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"GOOD** AND TOOR" WRITING AND WRITERS: 
STUDYING INDIVIDUAL PERFORMANCE AS A PART OF TEST VALIDATION 

Desmond Allison and Evelyn Cheung^ 
University of Hong Kong 



1. Introduction 

Our paper sets out to investigate issues of test validation by placing them in a context. It 
examines the ways in which instances of "good" and "poor" writing have been identified in the 
marking of part of a writing test that is given each year to incoming Arts Facultv students at the 
University of Hong Kong. It also looks at overall performances of "good" and "poor" writers. 
Implications concern content validity of the test and uses to which its results are put. 

1.1 Assessing Writing in English for Academic Purposes 

Our aim in this section is to establish a working perspective for our study. We do not 
attempt to review the large and growing body of literature on assessment of written English, but 
confine ourselves to commenting on some familiar questions that are also important for our 
purposes. 

The much-discussed tension between reliability and validity (Davies 1978; cf. Morrow 1981: 
13) in language testing is highly evident in the assessment of writing. Let us assume a context in 
which second language learners are preparing to use English for study purposes, over a wide range 
of academic subjects. To test the writing abilities of these learners, whether for initial placement 
purposes or as end-of-course assessment, it is necessary among other things to ensure that (i) a 
representative range of abilities and tasks is covered; (ii) test marks and grades arise from criteria 
that are clear and not arbitrary; (iii) information from the test can be interpreted as needed (e.g. for 
purposes of course evaluation, or ror statements about a learner's writing abilities). These issues of 
principle raise many problems, and have also to be reconciled with practical constraints in the 
testing situation. 

a. Coverage 

The issue of what abilities a writing test is testing calls for both a priori and post hoc 
validation of test content and of the underlying construct. In a useful discussion (not confined to 
tests of writing). Weir (1988: 21-25), following Anastasi (1982: 132). affirms that content validity 
depends upon the relevance of the individual's test responses to the abilities a test seeks to 
measure. Simple inspection of test content cannot suffice to establish content validity.^ Weir also 
points out that, while statistical data can certainly contribute to our awareness of the construct 
underlying a test, such data do not come with ready-made conceptual labels. Instead, findings need 
to be interpreted, in the light of whatever theory we have at our disposal. 

The specific abilities or knowledge that a writing test may measure will depend, then, both 
on the range of tasks set and the ways in which individuals respond to these tasks. We would like 
immediately to draw attention to work by Gumming (1989) that offers a two-factor account of some 
second-language writing test data in terms of (i) L2 proficiency and (ii) writing expertise. "Writing 
expertise" here relates to qualities of discourse organisation, and of content selection and control in 
response to (complex) demands of the writing task, and is distinct from lexicogrammatical 
"proficiency". It is relevant to our discussion to ask how far any test of written English in a second 
language context draws, and should draw, on each of these hypothesised traits. 
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b. Marking Criteria 

Designing representative writing tasks that can also be reliably graded is clearly a 
challenging activity. For one thing, concern for reliability cannot be satisfied by eliminating 
subjectivity from the assessment procedure. Gannon (1985: 39) argues convincingly (in our view) 
that objective assessment of writing can only be achieved by limiting oneself to trivial or very detailed 
aspects of writing. We believe Gannon's line of argument holds good in a second-language context, 
especially if we aim to test writing expertise as well as English language proficiency. 

How can subjective marking be made reliable and valid? Current wisdom in this domain 
(also in oral testing) appears to promote the use of rating scales that draw upon a complex range of 
criteria yet assign a single mark on tho scale. For example. West (1989) presents scales used by 
two leading examining boards in Britain - the Cambridge Syndicate Marking Scale and Oxford 
Delegacy Scale. West claims that the use of these scales helps examiners not only to ensure inter- 
marker reliability on one task but also to "overcome one of the great problems when testing writing: 
how to compare scripts from different tasks?" On the other hand, White (1985: 18-33) advocates 
that "holistic scoring" be carried out according to criteria that are developed for each particular essay 
question and which are interpreted according to the particular group that is tested.^ 

c. interpretabillty 

If rating scales are interpreted very differently, then reliability problems may persist. 
However, high reliability in itself does not guarantee validity. High agreement among raters can still 
leave us with problems in suggesting just what constructs underlie the written performance - and 
indeed the ^issessments by different raters. 

1,2 Assessing Writing in One Context 

We shall pursue these language testing concerns in the context of a group of tests that are 
administered annually by the Language Centre of the University of Hong Kong to some 400 incoming 
students of the Faculty of Arts. These tests are given in what are called 'Language Analysis 
Sessions', and are generally known as "the L.A.S. tests". Separate tests are given for each of the 
"four skills" (For an outline of the four tests, see Fok 1985: 127-129). 

Our focus is on the L.A.S. Writing test, and in fact on one of this test's five sections or 
'subtests'. An account of the LA.S. Writing test and of its rationale, which relaies to the demands of 
a tutorial paper on students' writing abilities, can be found in Low (1982). While various 
modifications to the test and marking scheme have continued to be made, the basic form of the test 
remains unchanged. 

In brief, the test provides for continuity of topic across a range of tasks. The theme of the 
paper is on the changes brought about by modernization in a small Chinese village called Ma Lin. 
The five subtests and tasks are: 

1. Proofread (the introduction, which sets out terms and scope of the paper). 2. Write text (from 
pictures): describe change over time (what has happened to the village). 3. Write text (from note 
cards); 4. Structure an argument (multiple choice): apply two viewpoints concerning change to 
sample village. 5. Edit text (a draft conclusion) in light of a reader's comments (re formality and re 
validity of conclusions). 

We shall describe subtest 2 in a little more detail, for reasons that will become clear. In this 
subtest, the student has to demonstrate his/her linguistic and rhetorical ability to handle the notion 
of change. Minimal input to the writing task is provided through graphic information showing an 
agriculturally based village beside the sea. The second picture Is rather different, with the area 
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looking more industrialized and populated. In both diagrams, the pictures are labelled to prevent 
any misinterpretation of what they are supposed to symbolize. 

The marking scheme for the L.A.S. Writing test is fully objective for subtest 4 (multiple 
choice); almost so, for subtest 1 (proofread for error in each line of the text); and subtest 5 (edit 
parts of the text that have been marked). Alternative possibilities over the years for subtest 1 and 5 
have been itemized as acceptable or not. Subtests 2 and 3 involve impression marking, guided by a 
set of criteria. Four of the five subtests are marked by single markers following the marking scheme; 
subtest 2 is the exception. 

For subtest 2, a total of four markers are Involved (in some years six markers): two pairs of 
markers with each pair marking half the scripts. Markers' meetings before the test allow each pair to 
discuss a number of sample scripts that they have marked and to ensure some comparability of 
standards. Actual test papers are compared in cases of serious discrepancy only, as the timescale 
for the whole operation is exceedingly tight. 

Subtest 2 appears to be the most open-ended. Despite a possible challenge from subtest 3, 
subtest 2 is in our view the most "writing-like" part of the Writing Test. As we shall see, the subtest 
has also given rise to some problems during validation studies. We decided to explore this subtest 
further. 

1.3 Validation of the Writing Test 

Validation of this test hitherto has been in terms of statistical analysis and interpretation of (a) 
test results; (b) test-taker attitudes as measured by a post-test questionnaire. Work now planned or 
in progress aims to supplement and enrich these analyses through (c) more attention to individual 
test performances (this paper being one contribution here) and (d) interviews with test-takers 
(planned for 1990). 

Without going into great detail, we would like to comment on a few of the findings reported 
by Lee and Low (1982) with respect to test results. 

Lee and Low report a two-factor solution for their test results. They interpret this solution in 
terms of the relevance of grammatical conskJerations (but not of extended context) for the two 
subtests (1 and 3) that load highly on Factor 1 , and of the relevance of extended context (but not of 
a focus on grammnr) for the two subtests (4 and 5) that load highly on Factor 2. We have earlier 
remarked upon the difficulty of assigning conceptual labels to the outcome of a statistical analysis. 
Let us note here that the relevance of "extended context" to test-taker responses cannot be 
established solely by pointing to the fact that subtests 4 and 5 occur in the latter part of the test 
paper. 

The omission of subtest 2 from the account is worth noting. Lee and Low report that subtest 
2 "had a high error variance (.24) and behaved in a strange way". They therefore decided to omit it 
from further calculations in their factor analysis. Low (1982: 256) also admits to "some problems 
with inter-marker reliability" for this subtest, although the reliability coefficient (alpha) of .76 reported 
by Lee and Low (note 2) appears to us to be respectable given the nature of the task. 

We suggest therefore that subtest 2 is an interesting instance of reliability-validity tension. 
The test appears to be a valid writing task for the testees, but has some reliability problems that limit 
validity. Since the Lee and Low study, there has been revision of a few Items in subtest 4 and 5 and 
minor changes In the wording of the instructions in subtest 2. The overall rationale of the test 
however, has remained the same. An attempt to factor analyse the 1986, 1987, 1988 and 1989 
results, following the procedures adopted by Low (Principal Component Analysis with varimax 
rotation), suggests that the test Is giving us different information now. The results will be discussed 
later in another section. 
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2. Aims of the Study 

The overall aim is to extend the study of performance on LA.S. Writing Subtest 2, and to 
analyse it in terms of the Writing test as a whole. On the face of It, subtest 2 is a more direct test of 
writing ability. For this very reason, it may give rise to some of the difficulties incurred by standard 
essay questions, which include the difficulty of telling "what any given mark actually means** (Low 
1982: 51). 

The main aims of the study are to look into individual performance in greater detail, in order 
to throw more light on the question of what given marks may mean. We are particularly ir»terested in 
the wi Iters who were identifiable, in terms of their subtest 2 performance, as being well above or well 
below average. We shall henceforth use "good" and "poor", in double quotes, as labels for two 
groups of writers that are defined operationally in each year by the subtest scores (Please see 
Methods section for details). Whether, to what extent and why these value labels are actually 
appropriate is what our study is all about. 

Two broad questions are: What (possibly quite varied) features of writing could be identified 
as characteristic of (some or all of) these "good" and "poor" writers? And how did the individual 
writers, whether "good" or "poor", actually fare in terms of the writing test as a whole? 

The choice of lexical, grammatical and discourse features of writing to be examined will be 
taken up in the Methods section. 

A final and very important aim is to evaluate our findings. We shall seek a better 
understanding of what abilities our chosen subtest actually measures; of whether it does so as 
clearly and reliably as one would wish; and of whether abilities that are not receiving credit in the 
marking of the subtest should be given more attention in the marking scheme (either within the 
subtest or elsewhere in the Writing test).^ 

3. Methods 

3.1 Choice of Scripts 

We decided to work with selected scripts from two years (1987 and 1989. Plans to include 
1988 scripts were set aside because of time restrictions). 

As ours is not primarily a statistical study, we are not concerned with equivalence of sample 
size either within one year or across the two years in the study. Instead, we have chosen to 
consider as "poor" all scripts receiving the lowest mark on the rating scale (a five point scale) from at 
least one marker. In practice, this means that the other marker had either also given the lowest 
mark or a rating one interval above the lowest mark. Similarly, we define "good" as receiving the 
highest mark from at least one marker (and, in practice, the highest or highest but one mark from 
the other marker). 

This initially yields figures as follows (with 1988 also included): 
Table 1 : Total number of "Good" and "Poor" scripts 



"Good" scripts 



"Poor" scripts 



1987 
1988 
1989 



8 
12 
10 



40 
24 
20 
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It appears generally to be common practice that examiners prove reluctant to use the top 
band on a scale, even when they are encouraged to use the full range in their marking (North 1990). 
In the present study, the imbalance between "good" and "poor" for 1987 was so great that we 
decided first to select scripts which received the lowest mark from both markers (17 scripts) and 
then select at random another 6 scripts which received the lowest mark from one marker, with the 
second marker giving the mark which is one interval higher. All "good" scripts for 1987, and all 
scripts in both categories for 1989, were included in the study. 

For reference in this paper, scripts are coded by year (87 or 89); by category (G = "Good", 
P=''Poor"); and by an arbitrary number for each individual. Since students do not have access to 
their subtest grades, confidentiality is preserved by this procedure. 

3.2 Descriptive Categories 

Determining what these categories should be formed an important part of the study. As for 
testing, so possibly for research itself: objective categories are demonstrable but can be trivial, while 
more in-depth qualitative assessment of individual writing, however perceptive, will raise problems of 
verifiability or agreement. Taylor (1986) makes a strong case for detailed study of individual scripts 
(which he exemplifies) rather than error surveys. For test validation purposes, however, one would 
need to study quite a number of individuals and still seek some form of generalisability of statement. 

We therefore decided to look at the selected scripts using objective measures; to adopt any 
qualitative assessment where appropriate; and to follow this by a descriptive account of 
generalisable features, with examples for illustration. For convenience, we list the categories under 
three headings: lexis, grammar, and discourse. 

3.3 Objective Measures 

a. Lexis 

We decided to do the following: 1) count the number of words (word tokens) as a measure 
of text length 2) count the number of word types ( eg. 'was'; 'are' are counted as one type) 3) 
calculate type/token ratio as a measure of lexical repetition. 

b. Grammar 

The T-unit is adopted as a measure of syntactic complexity (one main clause with any 
subordinate clauses). We decided to count the number of T-units; the number of error-free T-units; 
the average length in words of T-units and of error-free T-units. Length of error-free T-units is 
typically a better predictor of test performance (Larsen-Freeman 1978; LIm 1983). 

c. Discourse 

We decided to count the number of paragraphs and note the presence of a recognizable 
introduction and conclusion. 

3.4 Qualitative Assessments 

We are interested also to assess the quality of the introduction, conclusion, as well as the 
'development* of ideas throughout the piece of writing. An Impression mark was given using a 3- 
point scale: 'good'; 'OK'; 'poor*. Zero mark was given for the absence of an introduction and 
conclusion. (By these, we mean some introductory or concluding statements at what we took to be 
a suitable level of generality). 
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3.5 Statistical Correlations 

We are interested in the simple correlation between the five subtests; and between the total 
writing rnark and each of the subtest for 1987 arxi 1989. Simple correlation was also computed 
between the tvyo marks for subtest 2. As mentioned earlier, factor analysis was performed on the 
five subtests for the four years from 1986 to 1989. 

It is clear to us that objective and evaluative information needs to be integrated so that 
useful and meaningful statements can be made. With regard to lexis, type/token ratios have sen/ed 
to draw our attention to individual scripts that have comparatively frequent instances of repetition. 
Effectiveness of such instances has been evaluated, and one or two cases are reported below. 
Another one of our descriptive aims is to focus on a small group of lexical items that are related to 
the notion of change, which is central to subtest 2. We decided to look at how the lexical items 
•^change"; "replace"; "turn"; and "develop" are used in each script. This involves grammatical 
considerations. We are also interested, at the discoursal level, to look at how connectors are used 
to link ideas together. We shall report on all this in the Discussion section. 

4. Findings 

4.1 Quantitative Survey of Scripts (Table 2; Table 3) 

Although the mean scores are fairly crude measures, they tend to support existing research 
and teacher comments that "good" writers on average would write longer texts than "poor" writers, 
and would make fewer errors. The difference in the average length of error-free T-units appears to 
be more pronounced than that in the average length of T-untts in the two groups in 1989, but not in 
1987. We are aware, of course, that the unequal number of samples between the "good" and "poor" 
scripts inhibits us from making too strong a claim about trends that have emerged. 

The difference in the number of paragraphs appears to be negligible, as both groups tend to 
use more than a paragraph. It is difficult to interpret the average mark for introduction and 
conclusion in table 2, because the number of students in 1987 who wrote an introduction and a 
conclusion is small. However, it is worth noting that more students in the "poor" group (14 out of 23. 
or 60.9%) wrote a conclusion compared to the "good" group (2 out of 8 or 25%). The average mark 
for 'development' again seems to show that the "good" group scored higher. 



Table 2: Mean scores of 1987 scripts 



L(W) 


T/T TU W/TU 


EFTU 


%EF 


W/EFT 


Pa 


Int 


Con 


Dev 




Good 210 


.58 15.3 14.1 


11.3 


73 


13.3 


2.1 


1.0 


0.5 


2.5 




Poor 171 


.58 15.3 11.4 


5.7 


38 


10.8 


2.0 


0.5 


0.8 


1.4 




Table 3: Mean scores of 1989 scripts 
















L(W) 


T/T TU W/TU 


EFTU 


%EF 


W/EFT 


Pa 


Int 


Con 


Dev 


Good 204 


.56 14.4 14.7 


11 


78 


14.7 


2.7 


1.2 


1.0 


1.8 


Poor 163 


.56 12.7 13.0 


3.9 


32 


10.7 


2.2 


1.0 


0.6 


1.3 
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KEY: L(W): length in token words; T/T: token/type ratio; TU: T-unit count; W/TU: average T-unit length in words; EFTU: 
•rror-free T-unit count; %EF: percentage of error-free T-units; W/EFT: average length of erroMree T-unit; Pa: 
paragraph count; Int: mark given to introduction; Con: mark given to conclusion; Dev: mark given to 
development. 



4.2 Percentile Ranking 

For 1987, 5 of the 8 "^good" scripts are the work of the students above the ninetieth 
percentile. Two scored above the eightieth percentile and the last one near the margin of the 
eightieth (79.5). As for the "poor^ scripts, 7 of the 23 scripts were the work of students scoring 
below the tenth percentile. Of the remaining 16, one scored above the sixtieth percentile; two more 
above the fortieth percentile; another three above the thirtieth percentile; three more above the 
twentieth percentile; and seven more above the tenth percentile. 

For 1989, 8 of the 10 "good" scripts on subtest 2 are the work of students scoring above the 
ninetieth percentile on the Writing test (including the two highest scorers on the test). The other two 
students were ranked at the 87th and 79th percentile. As for the "poor" scripts, 13 of the 20 scripts 
were written by students scoring below the tenth percentile (including the two lowest scorers on the 
test). Of the remaining ca'^.es, one scored above the eightieth percentile; a second also scored 
above the fortieth percentile; a third above the thirtieth percentile; and two more above the twentieth 
percentile. 

it appears that if a student does well on subtest 2, it is highly likely that s/he would perform 
well on the test as a whole. However, with the "poor" scripts, the picture is less clear. While about 
60% of the students in both years rank below the bottom twentieth percentile, the rest could range 
above the twentieth to the eightieth percentile. 



4.3 Simple Correlations 



Table 4: Overall correlation between mark 1 and mark 2 for subtest 2 



1987 


.72 


(P < 


.01) 


1988 


.59 


(P< 


.01) 


1989 


.63 


(P < 


.01) 



Table 5: Correlation among the subtests; and between the writing total and the subtests for 1987 



Subtest 


1 2 3 


4 


5 




1 


.32 .25 


.37 


.34 




2 


.24 


.27 


.15 




3 




.17 


.09 


(P < .01) 


4 






.27 


5 










Total: 


.70 .62 .63 


.62 


.57 
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Table 6: Correlation among the subtests; and between the writing total and the subtests for 1989 



Subtest 


1 


2 


3 


4 


5 


1 




.42 


.37 


.36 


.22 


2 






.29 


.24 


.15 


3 








.18 


.16 


4 










.32 


5 












Total 


.72 


.64 


.64 


.64 


.60 



(P < .01) 



4.4 Factor Analysis 

Table 7: Results of factor analysis of the writing test (varimax rotation) for 1987 



1987 

Factor 1 Communality Percentage of Variance 



Subtest 1 .757 .573 40.2 

Subtest 2 .624 .389 19.1 

Subtest 3 .500 .250 15.3 

Subtest 4 .684 .468 13.7 

Subtest 5 .574 .329 11.8 



Table 8: Results of factor analysis of the writing test (varimax rotation) for 1989 







1989 






Factor 1 


Communality 


Percentage of Variance 


Subtest 1 


.775 


.601 


42.1 


Subtest 2 


.662 


.439 


19.4 


Subtest 3 


.612 


.375 


14.9 


Subtest 4 


.648 


.420 


13.1 


Subtest 5 


.519 


.270 


10.5 



Although the results of four years from 1986 to 1989 were factor analysed, only the results for 1987 
and 1989 are shown here. In the four years, only one factor could be extracted. (This remained the 
case when the analysis was repeated after excluding subtest 2, to ensure exact comparability with 
the procedure followed by Lee and Low (1982)). 
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S. Discussion 

5.1 Subtest 2 and the L.A.S. Writing Test 

The correlations between the two sets of marks for subtest 2 (Table 4) indicate that obtaining 
high inter-marker reliability remains a problem. Interestingly, this problem persists despite revision of 
the marking scale. Although "rough justice" is done in that all marks given on the five-point scale 
correspond to within one band (i.e. the second mark is the same as, or one above, or one below the 
first mark in any case), it appears that markers are operating with rather different practices that may 
reflect differences in their constructs of what constitutes "good" or "poor^ performance on the task. 
Further study of this issue would have to examine scripts taken from "the middle" of the distribution. 

Results from the factor analysis for the four years suggests that there is a common factor 
bet/.een subtests and all other factors are specific factors. Each subtest therefore can be assumed 
to measure rather different aspects of language skills. In the four years, subtest 1 is the most 
predominant. The results here are supported by simple correlations in Tables 5 and 6. The writing 
total correlates positively and significantly, although not very highly, with each of the subtests. 
However, there are very low correlations among the subtests. 

It was mentioned earlier that it was easier to predict a higher total mark for students who 
performed well in subtest 2, but that the converse does not always hold among the "poor" scripts. 
What seems to stand out now is that subtest 1. which is the explicit grammatical component of the 
test is comparatively a better predictor of the overall writing test score. (Note the higher correlation 
coefficient for total and subtest 1 in both years). Similarly, analysis of the scripts of subtest 2 also 
suggests that the "good" scripts have comparatively fewer lexicogrammatical errors (Tables 2 and 3). 

Part-whole correlations and inter-subtest correlations do not suggest that subtest 2 is 
markedly unusual (though admittedly this information is not very refined). The tendency for 
moderate correlations with subtest 1 may reflect an element of "knowledge of the L2/grammatical 
accuracy" in the marking of subtest 

5.2 Measures of Individual Performance 

Comparison of the mean scores for "Good" and "Poor" categories in the same year gives few 
surprises. With the exception of type/token ratio (and of identical number of T-units per essay in 
1987), means are consistently higher for the "Good" group. The difference in both years is most 
marked for absolute numbers of error-free T-units (EF=Ts) per essay. The status of 'average length of 
EFTs' as a better predictor than 'average length of T-units* for performance on the writing teot (cf, 
Lim 1983: 42) receives support from our 1989 but not our 1987 data. 

Type/token ratio is the same ff both groups in 1987 and also in 1989. The range, however, 
is greater for the "Poor" group, from a low of .49 to a high of .70 (two cases) in 1987. The same 
holds true for 1989, with a range of .44 to .66 (two cases). This reflects two kinds of text: scripts of 
length comparable to the "Good" group but with rather less lexicogrammatical variety (ie. with more 
repetition) and hence a low type/token ratio, and very short scripts in which there is less lexical 
repetition and hence a high type /token ratio (It is worth noting that grammatical as well as lexical 
considerations affect outcomes here). 

We note that type/token ratio is not an effective measure of expressivity or of explaining why 
one text is better than another in terms of lexical choice (Carter, 1987: 89). However, we find that 
the type/token data provides an additional reason for examining and comparing certain scripts, and 
for checking somewhat unusual features of individual scripts within the "poor" and "good" sample. 
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While there may be a tendency for teachers to approve of lexical variation (suggesting a 
wider range of available vocabulary), and while this is to some extent reflected in the marking 
scheme (e.g. ways of signalling "change"), it should be stressed that lexical limitations appear not so 
much in the amount of repetition as in the uses to which repetition is put. In one "poor" script 
(87P21), the word Village" is repeated nine times. In this case, the repeated use of the word did not 
help to clarify the situation being described. Its overuse created a sense of redurxiancy instead. A 
"good" script (87Gl) however, with a somewhat similar type/token ratio, used the word Village" 
seven times, (in a comparatively longer text, 261 words compared to 163), and used the word rather 
more effectively. 

The walled village which was the people's main dwelling In the 
past has become a model village mainly for tourist attraction. 
The village has become a modern town with various economic 
activities. 

In the three instances here, the lexical repetition provides a framework for a developing argument. 
Each use of Village" is distinctive. The former Vvalled village" is now a "model village", and "the 
village". Ma Lin has become a modern town. The repetition here contributes to coherence. 

The listing of the lexemes of "CHANGE" is to some extent arbitrary, (since there are many 
indirect signals, such as tense shift or lexical contrast that were not looked at) but the procedures 
are consistent and still informative. The lexemes change , replace , turn and develop in the context of 
subtest 2 share this notion of 'current relevance' and of a state of affairs moving gradually from one 
state to another. Most students used the word change in various grammatical forms in their text and 
to a lesser extent, replace , tum and develop . 

Examination of the occurrences of change as a verb among the "good" scripts in 1987 and 
1989 show that in all twelve occurrences (100%), it was used in the present perfect tense. Among 
the "poor" scripts this tense accounted for 12 occurrences out of a total of 37 (32.4%), and the rest 
were either in the past simple, present simple, past perfect (once) or present progressive (once). 
The notion of change in subtest 2 is much more adequately captured using 'dynamic' verbs, which 
include the few above, in the present perfect form. As far as the word 'change' is concerned, the 
"good" scripts are consistent in using the present perfect tense, and are probably rewarded for it. 

It has long been recognized that improving the range of vocabulary of a second language 
learner is most effectively done in context, so that the collocational field of the word is also 
perceived. Our feeling is that while lexical variety is evident in most of the scripts, errors in 
collocation (and grammar) meant that lexical range per se did not necessarily receive credit from 
markers. 

It is revealing to find that "good" students as well as "poor" students produced a number of 
anomalous sentences, as in the examples below : 

(87G1) ... rice fields have been changed into large 

and modern factories. 
(89G10) Th^ rice fields on the hillside have been 

changed into factories. 
(89G6) ... the fields and gardens have turned into 

factories. 

(89G2) ... the rice fields have been replaced by 
noisy factories. 
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Writers of both the "poor" and "good" scripts in the sample do not appear to be sensitive to 
the inappropriacy of seeing rice fields turn into factories. A possible explanation for this recurring 
feature may be traced to interference from L1. Direct translation equivalents in Chinese prove to be 
more acceptable. 

With regard to grammar, our findings on average length of T-units also drew attention to 
unusual scripts in each of our categories. There was one striking instance {89G10) of a "good" script 
that exhibited quite high control over a very limited grammatical range (no major subordinate 
clauses). Conversely, some of the "poor^ writers attempted more complex grammatical structures 
that they proved unable to control, as in the following example. 

(89P16) As mentioned above, we will have a case study on Ma 
Lin, as an example, to show have modernization has changed an 
traditional agricultural village to an industrial town from 1950s to 
the present. 

We did not attempt any systematic categorization of grammatical errors within the time 
available to us. We noticed that form class errors were frequent among the "poor" scripts (e.g. "an 
industry town"; "agriculture life"; "agricultured"), but were rare among the "good" scripts. 

An attempt at describing the discoursal competence of the students was through evaluating 
the introduction, the conclusion and overall development in the sample texts. We interpreted 
introduction to mean any 'advance organizer* which gives an overall perspective of change seen 
from the present or in transition and/or which referred to Ma Lin as an example of the process of 
change. The instructions regarding subtest 2 are explicit about the purpose of the task, which is to 
describe physical changes and then socioeconomic changes, but it is left open to the students how 
they would like to structure the text. In both years only about 50% of the scripts contain 
introductions. Introductions which merited a *3* are not only found among the "good" scripts, but 
also among the "poor" ones. In the following example, takf n from among the "poor" scripts, the 
student demonstrates that he can write an introduction, buv grammatical inaccuracies and poor 
development of ideas in the rest of the text possibly affectcjd his final mark. 

(89P8) Ma Lin, an agricultural village in the 1950's. But now, it 
become a tourist attraction. Let us see how does this village 
changed in this short period. 

See also the example (89P16) we cited earlier as an instance of attempted grammatical 
complexity. 

Similarly, conclusions are found only in about 50% of the scripts in both years. Compared to 
introductions, tlie quality of conclusions is lower. This seems to reflect a general trend among both 
"good" and "poor" scripts. 

Discoursal skills are not only demonstrated through elaborate advance signals at the 
beginning of the text or through a satisfactory concluding statement. Achieving coherence is 
attempted through a number of ways. Some students rationalize the changes in Ma Lin through a 
chain of cause and effect events. Others depend heavily on the use of connectors to move from 
one subject to another. "Good" scripts generally cover the topic adequately, with very few irrelevant 
elaborations. There are many cases of misuse of connectors, especially 'moreover' and 'on the 
other hand' in both "good" and "poor" scripts which warrant further careful study. An example to 
illustrate this can be found in a "good" script. 
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(87G2) In the early 1950'8, Ma Lin was only a traditional village. 
People living there mainly engaged in the primary activities such 
as fishing since Ma Lin is near to the coastal areas. Besides , 
with the presence of lowland, so the villagers also engaged in 
the activities of market gardening, that was, the growing of 
vegetables and flowers as well as growing paddy .... On the other 
hand , Ma Lin was a walled village since it locates near the 
coastal areas, thus the whole village was surrounded by the wall. 



There seems to be & higher marker tolerance for inadequate signalling inter-sententially, as 
long as ideas flow reasonably smoothly in the overall comparative description of change. Writers 
who made errors at this level and who compensated by not making too many grammatical errors 
elsewhere are not penalized. 

The demands of this writing task upon discourse awareness and organisational abilities are 
not in fact particularly elaborate. Some of the writers explicitly organised their material in terms of 
such dichotomies as "economic" and "social" change, or of "physical" changes and changes in 
"occupation". Such overt structuring of text was by no means confined to "good" writers in the study. 
Indeed, some of the "poor" writer^^ were quite sophisticated in this respect. 

6. Conclusion 

What can be learned from all these observations and comments? At this point in our paper, 
we face the two dangers of offering rash and over-generalised conclusions or of venturing remarks 
that are so tentative as to be worthless. At the risk of folly, we shall state our conclusions firmly. We 
do so on the understanding that our investigation of how individual performance was assessed in 
this test forms part of a wider appraisal of the test's validity, and that the conclusions we propose 
are inferred from our observations rather than being demonstrated beyond all dispute. In addition, 
our comments on research methods remain impressionistic. 

Our conclusions relate to the content validity of the writing test in the study, the uses to 
which its results are put, and the implications of the study for test validation as such. 

The LA.S. Writing test provides a fairly wide range of tasks and may appear to achieve a 
reasonable overall compromise between demands of reliability arxJ those of validity. Our study 
shows, however, that the grading of individual scripts, on what is ostensibly a subtest concerned 
with continuous writing, in fact largely reflects the writers' lexicogrammatical proficiency in English. 
Ability to organise and develop a piece of writing - what Gumming (1989) considers as "writing 
expertise" - is not noticeably credited in the test. It is also clear that abilities to sustain a complex 
argument, e.g. in a term paper, are not assessed by the test. The distinction that the test makes 
between "good** and "poor" writing is consequently restricted in its scope. 

The extent to which this limitation is a problem for content validity depends partly on the 
uses to which results are put, since these uses are relevant to the choice of what to test in the first 
place. On the one hand, the "poor" writers identified by the test plainly need to develop their 
proficiency in written English. On the other hand, the courses to which Arts students have been 
assigned in recent years also place considerable emphasis on helping students to meet the 
discourse demands of academic communication. Yet discourse abilities or disabilities have not 
appreciably contributed to the placement of students in terms of these courses. The extent of 
individual learners' problems with local or global information structure in producing academic text is 
an issue that the test at present fails to address. At the very least, this discrepancy calls for careful 
evaluation within the Language Centre, both in terms of what we expect of our tests and of what 
student needs our courses should principally aim to meet. 
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Some of the methodological benefits of the study are intangible. Cleariy we have examined 
Individual scripts in more detail and more systematically than is normally possible when marking, and 
certainly when marking test scripts. This has given us a better feel for some of the qualities and 
limitations in the writing produced under test conditions. Among the practical consequences are that 
we can select scripts that would repay particular attention in markers' meetings. We can also 
document the persistence of reliability problems in marking. We can thus argue for a more 
sustained approach in future towards achieving greater marking consistency. There needs to be a 
more rigorous institutional procedure both for realising a marking scheme and for ensuring that 
interpretations of this scheme converge among individual markers. 

The study of individual test scripts has commended itself to colleagues and is being 
extended to validation of another writing test. We feel the approach could also usefully be applied to 
studies of how individual markers categorise scripts as particularly "good" or "poor**. 

The limitations of such work will be apparent. The choice of descriptive categories will 
depend on the researchers' evaluation of the writing task and doubtless on other considerations 
such as main research interests. While we make no apology for our own interest in lexis, we 
recognise that other researchers might have found useful information from a closer look at 
grammatical categories in our data. The danger here is that we all find what we are looking for, and 
overlook other potentially useful information. Provided, however, that work of this nature is 
undertaken to supplement rather than to replace other approaches, we maintain that it offers vital 
information in the validation of open-ended tests. 



NOTES 

1. Authors' names in alphabetical order. This paper was presented at the RELC conference on 
Language Testing and Language Programme Evaluation (Singapore. April 1990) and has been 
included in the ERIC database. 

2. For courses in English for academic purposes, we follow Hughes (1988: 42) in basing "content" on 
course objectives rather than on the syllabus and materials through which a course attempts to meet 
these objectives. The relevance of learner responses (processes as well as products) will be 
apparent in such a perspective. 

3. White also describes detailed procedures by which meaningful holistic marking can be undertaken 
successfully in one context. These procedures sound admirable, but are also cleariy demanding in 
terms of time and manpower. We plan to explore their feasibility within the Language Centre. 

4. With regard to abilities that may not receive sufficient credit, compare a suggestion by Low (1982: 
256. note 4) that more attention be given to the transitions made between different parts of the 
tutorial test paper). 
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Iniroduction 

In 1989 the Hong Kong Examinations Authority (HKEA) Introduced the new Use of English 
examination. The specific objectives of the examination were to: 

... test the ability of the candidates to understand and use English as 
might be required in tertiary education and/or future employment. 
(HKEA, 1987: 2) 

The examination sets out to test candidates' ability to: 

(i) understand and interpret spoken English as it might be 
encountered in academic and vocational situations; 

(ii) write clear, concise and grammatical English in an appropriate 
style; 

(iii) demonstrate both global and detailed understanding of a variety 
of written texts; 

(iv) integrate reading, writing and study skills in the pursuance of 
tc.^'<-based/prob!em solving activities. (HKEA, ibid) 

The examination was therefore designed to test candidates' readiness to use English in 
tertiary education and employment. Its development raised the question: would the examination 
results give sufficient information to determine whether students entering tertiary level education 
needed further English programmes to facilitate their academic studies? If so, there would appear to 
be no need to retest students at the beginning of their university career. 

The purpose of this study was to investigate the content and predictive validity of the new 
Use of English examination. More specifically, it set out to find out whether the examination could 
be relied on for determining students' language needs on entering tertiary education. Since prior to 
the introduction of the new UE examination institutions such as the University of Hong Kong (HKU) 
had developed their own placement /proficiency tests for this purpose, it was important to find out 
whether these internal tests would continue to be necessary. 

The comprehensive battery of tests developed by the Language Centre of HKU to assess 
Faculty of Arts students is known as the Language Analysis Sessions (LAS). These tests cover all 
four language skills and are therefore time-consuming to administer and mark. However, when the 
tests were validated in the mid-1 980's (Fok, 1985), they were found to have better predictive power 
than the old (pre-1989) Use of English examination in terms of students' end-of-first-year results in 
their academic studies. They were also found to be superior in terms of face validity, since they 
reflected more closely the language skills required of the students in their academic work. They 
were not only seen by the students but also by the academic staff of the Faculty of Arts as reflecting 
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the tasks demanded of the first year students. There was therefore every justification for 
administering the tests. 

There was also a further reason for administering the LAS test to students of the Faculty of 
Arts. These students were originally assigned to courses addressing specific language skill areas. 
For example, students found to be weak in spoken English but strong on reading and writing were 
assigned to an Oral course, while those weak in all skills were assigned to the Integrated Language 
Skills course. Experience showed, however, that arbitrary decisions often had to be taken as to the 
course most suitable for individual students as there was rarely a clear-cut distinction between those 
weak in one or two skills as opposed to ail four skills. On the basis of this finding, and the 
recognition that students, however good they may be in English, may still require English 
enhancement to maximize the benefits they accrue from their university studies, the Language 
Centre decided to streamline the courses it offered the Faculty of Arts students. At present it runs 
one course known as the English for Arts Students (EAS) course. This development is in line with 
current university thinking on English enhancement, which not only supports the extension of English 
programmes but is also moving towards making such courses credit-bearing. However, it weakens 
the case for testing on entry to university since the results are no longer needed for placement, but 
only for determining those students who are eligible for exemption from the course. 

Comparison of Tests in the Study 

In order to see whether the two tests under consideration, that is the UE and LAS, were 
indeed comparable, a detailed study of each subtest was undertaken. This, as can be seen from the 
brief descriptions of the subtests below, revealed that there appears to be considerable overlap 
across the subtests and the examinations as a whole, although there is no one-to-one 
correspondence between the subtests. There was therefore a sound basis upon which to proceed 
with the study. 

The New 'Use of English' Examination 

A fundamental change in testing philosophy underpinned the design of the new UE 
examination. Task authenticity became a primary consideration and is reflected in all four sections 
of the examination. 

Section A-Listening 

The Listening Paper sets out to test *the ability of candidates to understand, organize and 
interpret spoken English as used by educated arrJ fluent speakers of English as an international 
language' (HKEA, 1985: 2). The brief specifies that the spoken language used for the test is to be 
semi-formal rather than infc mal and that realistic contexts are to be provWed. Furthermore, the 
candidates are to use the information obtained from listening to accomplish specific tasks and the 
te!>t is to reflect real-life listening experiences. For example, candidates are allowed to listen once 
only to the information, but they are allowed to familiarize themselves with the task at hand before 
listening to the text so as to listen with a focus and purpose. An attempt is also made to cover a 
wide range of listening enabling skills, such as kientifying main themes and supporting details, 
extracting global information and recognizing speakers' attitudes. The 1989 paper included the 
presentation of a monologue and dialogue conducted in a business context, while the 1990 paper 
used a monologue conducted in the form of a semi-academic lecture. 

Section B-Writing 

This test sets out to assess the candidates' ability to write extended discourse of an 
expository nature, e.g. in the form of persuasive arguments, short reports, development of 
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hypotheses, etc. rather than straight narratives or descriptions. Candidates are given a choice of 
four writing topics from which they select one on which to write an essay of about 500 words. Care 
is taken to give each topic a realistic context and the writer a purpose for which to write. 
Assessment is based on organization of ideas and coherence of arguments as well as accuracy and 
range of the language used. 

Section C-Reading and Language Systems 

The reading part of this test, which carries about a quarter of the weighting of this section, is 
aimed at testing 'the ability of candidates to achieve an "in depth" understanding of an expository 
text' (HKEA, 1987: 3). The emphasis is not on fact-retrieval but on global comprehension of the 
overall structure of the text, which in turn depends on the candidates' ability to recognize the 
purpose of the text as well as the attitude of the writer. Items set on lexis require candidates to 
deduce contextual meanings through adequate interpretations of the text. 

The language systems part, which carries three-quarters of the weighting of this section, sets 
out to test 

the extent to which the systems of the English language have been 
internalized by the candidates: 

- the lexicon 

- the morphology 

- the syntactic relationship within and among phrases and clauses, and 

- the structural relationships among sentences within paragraphs 
and in discourse (HKEA, 1987: 3) 

This paper, which is to a large extent objectively marked, represents an attempt to tap the 
candidates' general language proficiency in English using test-tasks set at discourse level. The 
tasks, which include summary cloze, proof-reading, paragraph-building, list-matching, etc., call for a 
good mastery of English lexis and syntax as well as an awareness of cohesion and coherence 
features in English. 

Section D-Practical Skills for Work and Study 

As the title of this section suggests, the paper requires candidates to display such practical 
English language skills as are needed in work and study situations. A bank of information in verbal 
and graphic form is given to candidates as a data-file. Candidates are required to use the given data 
to complete several specified communicative tasks. In this way candidates' abilities in processing, 
selecting and presenting relevant information in a fitting manner as specified by the tasks are tested. 

The emphasis will be upon practical rather than academic skills as 
such, though the techniques and skills tested are those which are 
required for professional work which requires the ability to process 
information, make judgements and formulate solutions to problems; 
they will thus be equally relevant to students entering tertiary 
education. (HKEA, 1987: 4) 



Language Analysis Sessions 

Although HKU Language Centre's LAS battery of tests has been described elsewhere (Fok, 
1985) a brief description of all the subtests is given below for ease of comparison with the UE 
subtests. It is also worth noting that the ^ ibtests have undergone some changes over the last five 
years. 



ERIC 



17 

2Z 



HONGKONG PAPCRS IN UNGUiSTICSAND LANGUAG£ TEACHfNG U (1991) 



1. Listening 

The aim of this subtest is to test the candidates' ability to follow the overall development of 
academic discourse and to identify relationships between main points and examples. The test takes 
the form of an unscripted 35-minute lecture which has been recorded on video. The candidates take 
notes while viewing/listening to the lecture which they see once only. They then use their notes to 
answer short questions. 

The test is divided into four parts, respectively testing candidates' ability to: identify the main 
points of the lecture: correct certain inaccurate statements about the content of the lecture; 
recognize points made during the lecture by selecting those points from a list; kJentify the 
relationship between certain examples and the main points of the lecture. 



2. Writing 

This test is designed to test the candidates' ability to write a tutorial paper. It is divided into 
five sections and a common theme runs through them all. The candidates are given a certain 
amount of input in the form of notes of the type they are likely to collect from reference material. 
Each of the subsections tests a different enabling skill. Candidates are expected to proofread and 
correct grammatical errors within the introduction of a text; describe changes that have taken place 
over time; compare different theoretical views using information provided in the form of notecards; 
use appropriate connectives; reformulate sections of a text that have been marked in the margin as 
problematic in writing style. 



3. Reading 

This test is designed to test the candidates' ability to adopt appropriate reading strategies for 
dealing with a long piece of academic-type discourse. It tests the candidates' ability to understand 
the main ideas within different sections of the text as well as their understanding of specific 
information presented in the text. 



4. Oral 

This is a direct test of the candidates' ability to participate actively in a tutorial discussion. It 
takes the form of a simulated tutorial with a tutor leading a 15-minute discussion among four or five 
candidates. Before the actual discussion candidates are given time to read a short passage on an 
academic subject and they are given some questions to consider on the basis of the reading. 
During the discussion, the candidates are encouraged to interact with one another rather than merely 
respond to the tutor's comments and questions. They are assessed by the tutor as well as two other 
markers on the basis of the following criteria: fluency, accuracy and interactional skills. 



Procedure 

The results for UE, LAS and end-of-first-year subject examinations were obtained for 346 
first-year students admitted to the Faculty of Arts at HKU in September 1989. Details of these results 
are set out below. 
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1. UE Results 

These consist of four weighted scores, one for each section of the UE paper, namely 
Listening, Writing, Reading & Language Systems, arxl Practical Skills for Work & Study. These 
weighted scores when added together make up the total UE score (henceforth UTOT), for which a 
UE grade is assigned to each candidate. The University of Hong Kong requires a minimum of grade 
D for admission lo any of its programmes (In 1989, 46.3% of candidates sitting the UE exam 
achieved a grade D or above). 



2. LAS Results 

These are also weighted according to the four subtests, Reading and Listening carrying a 
weighting of 20%, and Writing and Oral a weighting of 30% each. They add up to an LAS total score 
(LTOT, henceforth). 



3. B.A. First Year Examination Results 

These are the results students received in their B.A. First Year Examinations in May 1990. In 
their first year students select 3 or 4 subjects from a choice of 17. Students' performance in these 
examinations determines their promotion to the second year of studies and their choice of major and 
minor subjects for their degree. 

These results were statistically analysed to investigate: 

a. Whether or not the new UE examination closely resembles the LAS tests in determining the 
subjects' overall language abilities on entry to university; 

b. Whether or not the new UE examination taps the same language ability factors as the LAS tests; 

c. Whether or not the new UE examination possesses a higher predictive ability than the LAS tests 
in terms of students' end-of-first-year results. 



The following statistical analyses were carried out on the results: 

a. UE total scores (UTOT) and LAS total scores (LTOT) were correlated to see whether the two tests 
yielded similar results. In addition, UE total scores (UTOT) were correlated against the LAS total 
minus the oral component (henceforth LTT), as oral skills are not tested in the UE examination (See 
table 1). 

b. Factor analysis was carried out to see whether there were distinct UE and LAS language factors 
or whether specific factors overlapped across the two tests, that is the UE and LAS (See table 2). 

c. UE and LAS results were correlated against the ervd-of first-year examination results to see which 
of the two tests performed as the better predictor of students' performance in their academic studies. 
In addition, correlations were run on the LAS total minus oral (LTT), since UE has no oral component 
(See table 3). 
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Findings and Discussion 

1. Con-elation between the UE Totals and the LAS Totals 

The correlation index between the UE total score and the LAS total score was .6117** (see 
table 1 below). Though statistically significant at .001 level, the correlation leaves almost two-thirds 
(64%) of the variance unaccounted for, and hence the significance is likely to be a result of the large 
sample in the study (n = 346). However, an important point to note is that the population used in this 
study is more homogeneous in terms of level of proficiency than the total population sitting the UE 
examination, since all the candidates in the study had a minimum grade D(10) in UE, that Is, they 
were taken from the top 40% of the total UE population. 

Since the UE examination has no oral component, UTOT was also correlated against the 
total scores of the LAS Listening, Reading and Writing subtests (LTT). This was found to be 
marginally higher at .6444** indicating that the oral subtest of the LAS did not significantly affect 
students' overall performance. It further suggests that the differences between the two tests cannot 
be accounted for by the lack of an oral subtest in the new UE examination. 

Table 1 : Correlation Indices for UE and LAS Total Scores 



Correlations 


UTOT 


LTOT 


LTT 


UTOT 


1.0000 


.6117** 


.6444** 


LTOT 


.6117** 


1.0000 


.8986** 


LTT 


.6444** 


.8986** 


1.0000 




1 -tailed signif: ** 


- .001 





2. Factor Analysis 

To obtain further evidence of whether the UE and LAS were tapping the same subskills, and 
whether higher correlations exist between subtests of similar skills than between the complete battery 
of tests, factor analysis was carried out. Factor 1 came out as a very strong factor accounting for 
35.5% of the total variance, whereas Factor 2 was much less significant, accounting for only 4.7% of 
the total variance. The individual factor loadings for the eight subtests are shown in Figure 2 below. 

When we look at the factor loadings for Factor 1. the strongest factor by far, we find no 
evidence of distinct UE and LAS factors. This is contrary to the findings in the 1985 study (Fok, op 
cit), and suggests that the present tests are not distinctly different in nature as was suggested of the 
old UE and the LAS in the 1985 study. There is also no clear evidence to suggest that similarity of 
language skills yielded significant overlap in the variance of the 8 subtests. Therefore one might 
conclude that Factor 1 is an Integrated language skills* factor, probably with slightly greater 
dependence on the reading and listening skills since the highest factor loadings are for LR, UR, LL 
and UL 

The highest factor loadings for Factor 2 are for LW (.62) and UW (.60) suggesting a writing 
factor over and above the integrated skills factor. However, since this factor only accounts for 4.7% 
of the total variance, the role played by writing per se in the two test batteries should not be 
overestimated. 
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Table 2: Factor analysis of subtests of UE and LAS 



Factor 1 



Factor 2 



UL 

UW 

UR 



.53 
.10 
.60 
.37 
.73 
.45 
.57 
.34 



.24 
.60 
.49 
.10 
.19 
.62 
.22 
.34 



UWASPS 



LR 
LW 
LL 
LO 



Key: 



UL - UE Listening 

UW - UE Writing 

UR - UE Reading & Language Systems 

UWASPS - UE Work & Study Practical Skills 



LR - 
LW- 
LL - 

LO - 



LAS Reading 
LAS Writing 
LAS Listening 
LAS Oral 



3. Predictive Abilities of the UE Exam and the LAS Tests 

From the correlation indices given in Figure 3 below, it can be seen that the LTT scores have 
the strongest predictive power, followed by the LTOT scores. If we take the B.A. First Year 
Examination as the criterion behaviour then it would appear that the IAS test results have more in 
common with the target performance than do the results of the UE examination. However, this 
difference in predictive power is much less marked than that found between the LAS and the old UE 
examination in the 1985 study (Fok, op cit). In addition, one must recognize the fact that there are 
many factors which contribute to the B.A. Examination results and that it would be impossible to 
identify them all so as to isolate the proportion of variance attributable to English abilities alone. 

If one looks closely at the ranking of individual "orrelation indices in each column, it can be 
seen that the LAS results conform more closely to our expectation as to which subjects require a 
higher level of proficiency in English. The LTT scores boast a highly significant .61** correlation with 
Comparative Literature, and a significant .47** correlation with English Language and Literature. The 
fact that significant correlations were obtained for such subjects as Political Science, Psychology, 
Geography and Geology, and History also offers no surprise as these are subjects which require 
students to write reasonably long English prose of a discursive nature. The fact that the LTT results 
produced a significant negative correlation of .27* against Chinese Language and Literature also 
seems consistent with the common-sense assumption that one does not need proficiency in English 
to do well In Chinese. 

The LTOT results yielded a similar rank order of subject correlations to LTT; again with 
Comparative Literature and English Language and Literature correlating most highly and Chinese 
Language and Literature correlating negatively. 

The UTOT results, by comparison, yielded less convincing figures with what may appear a 
puzzlingly high correlation of .59* (significant at the .01 level) for Statistics, the highest of all the 
subjects. This high correlation may. however, have been a result of the small number of subjects 
who took Statistics in the Faculty of Arts (n=:20). The fact that English Language and Literature and 
Comparative Literature still come rather high on the list while Chinese Language and Literature and 
Computer Science are found at the bottom with negative correlation indices restores si»bstantial 
credibility to the predictive power of the new UE examination. 
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Table 3: Correlations Indices between UE Exam Results. LAS Test Results and First Year Exam 
Results 



UTUT 




LTUT 




1 TT 

LTT 




CT 

ol 


.by 


UL 


.65 


CL 


.61 * 


CM 


.42 


EN 


.49 


MU 


.60 


rA 


.39 


kit 1 
MU 


.41 


EN 


.47** 


oo 


.38 


1 A 

JA 


.36 


oT 


.41 


OL 


.34 


DO 

ro 


icicle 

.35 


DO 

ro 


.38** 


MU 


.28 


oT 


.28 


t A 

JA 


O A 

.34 


JA 


.24 


SY 


.28* 


FA 


.30 


HI 


.22 


FA 


.27 


PY 


.29** 


PS 


.20 


GG 


.24* 


GG 


.28* 


PH 


.19 


CS 


.23 


HI 


.26* 


SY 


.18 


PY 


.22* 


SY 


.15 


GM 


.16 


HI 


.21 


PH 


.13 


PY 


.18 


PH 


.18 


EC 


.07 


EC 


.02 


EC 


.09 


CS 


.06 


CS 


-.21 


GM 


-.14 


GM 


-.16 


CH 


-.22 


CH 


-.25* 


CH 


-.27* 



1 -tailed signif: * - .01 ** - .001 



ST - 


Statistics 


n 


20 


EN - 


English Language Fx Literature 


n 


= 148 


FA - 


Rne Art 


n 


15 


GG - 


Geography & Geology 


n 


= 106 


CL - 


Comparative Literature 


n 


41 


MU - 


Music 


n 


13 


JA - 


Japanese 


n 


45 


HI - 


History 


n 


86 


PS - 


Political Science 


n 


= 120 


PH - 


Philosophy 


n 


89 


SY - 


Sociology 


n 


89 


GM - 


German 


n 


14 


PY . 


Psychology 


n 


= 178 


EC - 


Economic 


n 


= 145 


CS - 


Computer Science 


n 


16 


CH - 


Chinese Language & Literature 


n 


= 110 



TotalY N = 346 (students take 3 or 4 subjects in their first year at university) 
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Conclusion 

The above analyses seem to suggest that for the purpose of predicting prospective 
undergraduates' academic performance the LAS tests continue to function more effectively than the 
UE examination. This is not surprising if one bears in mind the fact that the LAS tests are targeting a 
much more homogeneous population, la crgme de la cr§me both in terms of English proficiency and 
overall academic achievement (.Although the LAS tests are a better predictor of undergraduates' 
first-year performance in their academic subjects, in some instances it would appear that the LAS 
tests would serve this purpose better if the oral subtest were removed from the test battery). 

Despite there being considerable overlap in skills tested, and there being a possible common 
Integrated skills' factor, the UE examination and LAS tests are not identical and could in no way be 
regarded as parallel. In order to maximise the information upon which decisions can be made about 
students' needs for English enhancement, it would seem necessary to continue with some form of 
internal pre-course testing. However, before any decisions can be made as to whether to retain an 
internal measure, one must consider the purpose of testing to see how far there is a real and 
justifiable need for the additional information provided by the LAS (or any equivalent) tests. 

Our study suggests that the new UE examination appears to be more valid than the old UE 
both in terms of face and predictive validity. It may be a sufficient measure in cases where English 
enhancement is not equated with remediation, but is considered an integral part of English-medium 
tertiary education to be offered to all incoming students. 
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TESTING LISTENING COMPREHENSION: A NEW APPROACH? 
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University of Hong Kong 



Introduction 

The aim of this paper is to discuss some of the issues involved in validating a listening 
comprehension test that was developed as part of a language proficiency measure for tertiary level 
students in China. The paper considers the need for validation, as well as some of the difficulties in 
validating a new subtest. It goes on to look at the rationale of the test in the Chinese context, and 
the theoretical basis upon which it was developed. It then describes the approach adopted, the 
results of piloting the test, and finally their implication for future test development. 

Test Validation 

Once a genuine need for a new test has been established, the test specifications have to be 
drawn up. These are frequently based on existing syllabi and/or needs analyses. However, as 
Alderson (1988) points out, both are subject to judgements being made by so-called experts, so the 
content of any test is dependent on the decisions made about what the learners need to know or be 
able to do with the language. Furthermore, test specifications have to be operationalized in a way 
that trivial items are avoided and items used are valid, that is, they test what they set out to test. 
Hence any test can only be as good as the operational realisation of its rationale, which, in reality, 
grows and is modified throughout the operationalization and cannot be fully developed in advance. 
In other words, the items of a test can only be validated against a given test rationale. 

The difficulty of constructing valid items may lead to a further problem, that referred to by 
Davies (cited in Morrow, 1981: 19) as the reliability-validity 'tension'. To increase reliability a tester 
may inadvertently take a limiting view of validity which results in a reliable test of somewhat dubious, 
though statistically acceptable, validity. 

Background 

The language tester's dilemma has nowhere been greater than in China where the number of 
testees and the difficulty of arranging the moderation of marking inevitably lead test writers to 
develop objective type tests and examinations. Such tests, despite all their recognised drawbacks, 
are favoured because they not only encourage testers to develop a large number of test items which 
allow for a spread of results, but also enable comparability of results across the country. It is 
therefore not surprising that major tests rely heavily on multiple-choice type questions. This can 
clearly be seen in the two tests, that is the Matriculation English Tests (MET) and the Graded Tests 
for English Majors (GEM), that are being developed at the Guangzhou Institute of Foreign Languages 
(GIFL)-one of the major test centres in China-for widespread use throughout the country. Both 
tests are designed as measures of English language achievement/proficiency, the former at 
university entrance level, the latter at various stages throughout the degree programme. 

The extei lal constraints that were imposed on MET allowed the test developers little room 
for experimentation with alternative forms of testing. However, this has not been the case for GEM. 
The test developers were given guidelines as to the skills to be tested and suggestions were drawn 
up as to test format, but considerable leeway was given during initial development (1988-89) when 
the first two GEM examinations (GEM 4, to be administered at the end of the second year of 
university study, and GEM 8, to be administered at the end of the final year) were written. 
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The GEM examinations which are made up of four papers attempt to balance the multiple 
choice format with other test formats, including open-ended writing tasks. In developing the 
examinations a conscious effort was made to ensure a positive washback effect of the tests on 
classroom teaching, yet at the same time the test designers were aware of the difficulties of marking 
the more open-ended questions especially once the tests had been piloted and were in large scale 
use. It was with such considerations in mind that the new listening summary cloze test, to be 
discussed in this paper, was developed. 



The Testing of Listening Comprehension 

The testing of listening comprehension has undergone conskJerable changes in recent years 
as a result of a greater understanding of the processes involved in listening (see e.g. Brown, 1990, 
Buck, 1988, Rest, 1990). There has been a movement away from the belief that listening 
comprehension is a one-way bottom-up process towards the theory that it requires a more complex 
combination of top-down and bottom-up processing. In other words, listening is not just the 
consecutive processing of subskills starting from the lowest level to the highest. It requires 
processing at a number of different levels to which no fixed order can be attributed. The listener 
utilizes not only his knowledge of the language being spoken, that is, his knowledge of the 
morphemes, phonemes, lexis and syntax of the language, but also his knowledge of the outside 
world and how it relates to the topic at hand, as well as his interpretation of what has been said so 
far, in order to comprehend the message. The listener hypothesizes about what the speaker will say 
next and uses his knowledge of the culture to help him understand linguistic complexities. He will 
therefore rely heavily on context and it is with regard to context that tests of listening comprehension 
have changed dramatically. There is now a recognition of the fact that "if we ask students to decode 
short decontextualized sentences, we are not testing listening comprehension at all but asking 
students to engage in a very unnatural activity which seems to be confined largely to the second 
language classroom", (Buck, 1988: 22). As a result, it is becoming less and less common to see 
decontextualized utterances being used in tests of listening comprehension. 

What appears not to have changed is the belief that testers should strive towards designing 
tests that are *pure\ Tests that require students to respond in speech or writing are considered 
* contaminated' because they are testing more than the skills of listening. Buck (1988: 33), for 
example, cautions against mistaking inten/ening variables and suggests ways to overcome the 
•problems' of contamination. Yet if we look at the skill of listening as it is used in reaNife, it will soon 
become clear that we rarely listen solely for pleasure or enjoyment. More often than not the listener 
is required to *do' something with what he has heard: to respond to it in speech, take notes on it, or 
reproduce it in written form. Researchers are beginning to assert the fact that students should not 
only be tested on their level of comprehension, but also on their ability to use the infornr^tion they 
have heard, even though they recognise that such more integrative tests may affect the reliability of 
the measure. 

Although judgements on the value of listening-skill tests primarily on 
the basis of statistical reliability are desirable on scientific grounds, 
such judgements are questionable in terms of educational 
principles....The unfortunate effect [of such tests] on the pedagogy 
(which is not necessarily intended by the writers) is that the test 
users-language teachers and learners-may come to accept that 
listening ability amounts to whatever can be measured reliably... (Post, 
1990: 180). 

Testers, therefore, should be willing to sacrifice test purity in order to maximize task 
authenticity and provide a positive washback effect on teaching. 
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Listening Summary Cloze 

One of the skills required of tertiary level students is the ability to understand and retain 
Infornnatlon imparted during lectures. The most commonly used means to achieve this end is 
through notetaking. Yet good notetaking in a second or foreign language is not an easy task. It 
may, in certain situations, be an important skill to teach and hence to test. Students studying 
English in China, for example, may be required to attend lectures on literature and culture given by 
native speakers of English or may want to go on to study abroad. They would benefit from 
notetaking being included as part of their syllabus. A similar argument would hold true for students 
studying in an L2 at tertiary level, as in Hong Kong. If the teaching is to be effective, then the skill 
needs to be tested. However, the testing of notetaking is not easy. Notetaking is a very individual 
skill which cannot be realistically assessed on its own. It can. however, be tested indirectly by 
asking students to take notes on a lecture but judging them on the accomplishment of a task based 
on their notes. 

A number of such tasks are in widespread use, varying from the more closed, objective 
tasks of completing an outline of a talk/lecture, to the more open, subjective tasks such as writing a 
summary of a lecture. The problem with the former is that it is not only unrealistic, but allows for 
only a limited number of test items. The problem with the latter, on the other hand, is the difficulty of 
the task as well as the marking. A desirable test would therefore be one that minimized these 
disadvantages, but at the same time had a positive washback effect on teaching. 

To meet these criteria the listening summary cloze test (LSC) was introduced as part of the 
GEM examinations. This test is made up of two tasks: 

1 . listening to a talk/lecturette and taking notes on it; 

2. completing a summary cloze passage based on the notes taken 
during the listening and notetaking phase. 

The first phase of the test, which is not assessed, is designed to facilitate completion of the 
second phase, and at the same time to encourage students to take notes while listening. The 
students use these notes to complete the second phase of the test, a summary cloze passage for 
which a rational deletion procedure is used. Deletions are placed on high information content to 
avoid students being able to complete the summary without having heard the talk/lecturette. One 
mark is awarded for each correct word or phrase used to complete the blanks. 

The LSC test described above is similar to the listening cloze (LC) test described in Buck 
(1988: 28-29) in that in both tests the cloze passage is based on a summary of the original text. 
However, in the LC test the students are given the summary prior to listening to the talk/lecturette 
and they can complete it while listening or at the end from memory, whereas in the LSC the 
summary is withheld until the testees have heard the text and completed their notes. Therefore, they 
have to understand the talk/lecturette to be able to take notes on it, and then they have to rely on 
these notes to complete the summary. 



Development of the Test 

Two 20-item listening summary cloze test were developed in 1989, one as part of GEM 4 and 
the other as part of GEM 8. The former was piloted with a total of 203 students-173 end of first year 
English majors and 107 end of second year English majors at GIFL. The latter was also piloted at 
GIFL, on 1 17 end-of-third-year English majors. There was some delay in piloting the tests and as a 
result fourth year students were not targeted during the first trial of the test. 
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The lecturettes used in GEM 4 and GEM 8 were on different topics. An additional difference 
was the order in which the information was given in the summary. In GEM 4, the summary followed 
the order of the original text, while in GEM 8 the order of the sumn^ry was different to that of the 
original text, presumably making it more difficult for the students to retrieve the relevant information 
from their notes. 

Results of Piloting 

After scoring the tests, the results were analyzed using SPSS. The results of the listening 
summary cloze were correlated with the results on the other subtests, including the rest of the 
listening subtest, as well as the total scores on the test battery. In addition, facility values and 
discrimination indices were calculated for each item. A summary of results is given in tables 1-4 
below. 



Table 1: Mean and Standard Deviation of the Listening Summary Cloze 





Items 


Mean (%) 


Std Dev 


GEM 4 (Yr 1) 


20 


8.3584 (42.8) 


2.7637 


GEM 4 (Yr 2) 


20 


11.4112 (57.1) 


2.8449 


GEM 8 (Yr 3) 


20 


9.7094 (48.5) 


3.6861 



Table 2: Correlation of Listening Summary Cloze (LSC) with a 30-item Listening Subtest and Total 
Test Scores 





Listening 


Total Test Scores 


LSC (Yr 1) 


.3865** 


.7478** 


LSC (Yr 2) 


.4644** 


.6783** 


LSC (Yr 3) 


.5679** 


.6650** 


1 -tailed significance ** .001 





Table 3: Summary of Item Analysis for Listening Summary Cloze (GEM 4) 



Year 1 




Year 2 




Difficulty 


Total 


No (<.3) 


Total 


No (<.3) 


Very Difficult 


1 


0 


2 


1 


Difficult 


6 


0 


3 


0 


Intermediate 


6 


1 


6 


0 


Easy 


4 


0 


5 


0 


Very Easy 


1 


0 


4 


3 
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Table 4: Summary of Item Analysis for Listening Summary Cloze (GEM 8) 



Difficulty 



Total 



No (<.3) 



Very Difficult 
Difficult 
Intermediate 
Easy 

Very Easy 



1 

5 

10 

4 

0 



0 
0 
0 
1 
0 



The mean scores indicate that the tests were of moderate difficulty. Not surprisingly, the 
second year students performed better on the test than the first years. A comparison of means 
across subtests is impossible since, as mentioned above, the two were quite different. 

The correlations with the rest of the listening test suggest that though there is significant 
overlap between the two, the listening summary cloze tests skills other than listening. In terms of 
test purity these tests can indeed be seen as contaminated, which was to be expected because of 
the dual nature of the tests. However, for both the second and third year results the correlations of 
the listening summary cloze with the listening subtest were higher than for the other subtests 
suggesting that the listening summary cloze was functioning to a large extent as a listening- test and 
not one of reading and filling in the blanks. The exception was with the first year results where the 
correlation between reading and the listening summary cloze was considerably higher than that for 
listening and listening summary cloze, the correlations being .6262** and .3865** respectively. This 
may have been because the first years are less proficient in the skills being tested since it is only 
once they start their degree programme that listening skills are intensively taught and hence they 
may still have to rely on reading skills to complete the test. This result was not unduly disturbing as 
the test was designed to be administered at the end of the second year and was somewhat difficult 
for the first year students. 

The item analysis is perhaps most interesting. Both listening summary cloze tests appear to 
have needed only limited moderation. For GEM 4 a maximum of 5 items needed to be looked at 
and possibly changed or omitted, whereas for GEM 8 only two items were unacceptable. Such a 
modest amount of moderation, notwithstanding Fletcher's claim that "in future the test needed to be 
constructed with more care" (1990: 66). was to be expected. 

On the basis of these results it was decided to retain the summary listening cloze as part of 
both the GEM 4 and GEM 8 battery of tests. The same test format has in the last two years 
(1989-91) been experimented with at the Language Centre of the University of Hong Kong with 
equally encouraging results. 



The listening summary cloze described in this paper appears to have a number of 
advantages. The talk/lecturette format allows the setter considerable flexibility in the choice of topic 
and the degree to which the topic is developed. The test may be made easier or more difficult by 
the length of the lecturette, the topic chosen or the method used by the speaker(s) to develop the 
topic. The lecturette can be purpose written and recorded as was the case at GIFL, or prerecorded 
from, for example, the radio, as is the practice at the University of Hong Kong. The latter is more 
authentic but depends on a bank of material being built up for possible future use for testing. 



Discussion 
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Another distinct advantage of this test format is that it lends itself to a large number of items 
being written. This not only eases the pressure at moderation since some of the items can always 
be eliminated if they do not work, but it also ensures a spread of niarks. The validity of the test 
depends on the validity of the texts chosen and the items deleted. In order to ensure that both 
lower and higher-order listening skills are tested, the rational deletion procedure is used. This allows 
the test setter to make judgements as to the most suitable items for testing and enables a balance of 
listening enabling skills to be sampled. However, it must be remembered that the testing of 
lower-order skills is easier and unless due care is taken to construct a balanced test, test validity will 
be sacrificed. 

The marking of the test items is largely objective, yet the testees are not restricted to the 
exact words or phrases used on the tape. This allows them to demonstrate their comprehension of 
the text and adds to the authenticity of the task. Furthermore, to complete the task the students 
need their notes; they therefore have a purpose for taking notes and they need not rely on memory 
for task completion. To test that the students can really use their notes, the setter can vary the order 
of information given in the summary. It is likely that following the exact order of the lecturette makes 
the task easier, but often in real life one has to be not only selective in the information to be used, 
but also aware that the order in which information is given is not sacrosanct. 

Like all objective type tests, the listening summary cloze is not easy to set. Setters have to 
be careful in choosing the listening text to ensure it lends itself to a good summary and that there is 
sufficient redundancy of information to allow for notetaking. They must also pay particular attention 
to the deletions to en?>ure that these cannot be filled in on the basis of general knowledge, on the 
one hand, and that tt(ey are not testing trivial information likely to be missed by the listener, on the 
other. Thus pretesting on colleagues who have not heard the tape is advisable, and the moderation 
of the marking scheme is vital. 

Careful administration of the test is also crucial. Clear instructions to the testees are 
essential, as they are for any test, and guidance as to the nature of the information to be noted may 
also be necessary. There are numerous ways testees can be prevented from filling in the sumnr^ary 
cloze while listening. The summary can be withheld until after the tape has been played or the test 
booklet can be prepared in such a way that the students are prevented from looking at the summary 
while listening. Whatever method is used, it is essential that the students do not fill in the gaps while 
listening as they are likely to miss essential information. 



Conclusion 

The validation of this test method is still in its early stages and more research needs to be 
undertaken. However, the results to date are encouraging and indicate the suitability of the test for 
use at tertiary level. When the GEM examinations go nationwkle in 1991-92 it is hoped that this test 
will have a positive washback effect on teaching and will encourage staff in China to move away 
from the more traditional approaches of teaching listening to more integrative ones which allow a 
greater variety of activity in the language classroom. It should also help students see the merit of 
good notetaking. 

The test itself, though an indirect measure of notetaking and listening, does appear to have 
face validity. It would be interesting to see if students coukJ fulfil the task of completing the 
summary cloze after a delayed period, such as a week, as this would verify that the students really 
are capable of using the notes they have taken during a lecture. 
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A considerable effort has to be made when writing the tests to ensure that there are no 
ambiguities in the summary and that the test is tapping relevant notetaking/listening skills. It would 
therefore seem advisable, even in a country as large as China where the problems of test security 
are well known, to build up a bank of tests and not to rely on having a new test for each successive 
administration. This would not only be a more cost effective approach, but also a more theoretically 
sound one. 

Finally a word of caution. There is no ideal test and the listening summary cloze is no 
exception. Although it is a test that appears to satisfy many prerequisites of a good test and 
therefore may well be suitable for advanced learners needing notetaking and listening skills, it should 
be considered in conjunction with other test formats and not as a substitute for all other listening 
tests. 
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Introduction 

Despite a substantia! body of research into reading problems among users of English as a 
second language, claims by teachers of English that such problems persist among ESL university 
students tend, in our experience, to be received with some scepticism. Learners themselves do not 
often rate reading as one of their major areas of difficulty, or as a desirable priority for courses in 
English for academic purposes (EAP). Subject teachers, course administrators and even some EAP 
teachers may reinforce beliefs that academic reading is not too much of a problem, at least not by 
comparison with listening, speaking or writing. Such thinking can influence the design of English 
teaching programmes and affect their potential outcomes. 

Experienced readers will have inferred that we believe the reading problems of ESL university 
students, in our own teaching situation, to be in some danger of neglect, and to be in need of 
attention. Interestingly, less experienced readers of academic writing might not have drawn any 
such inference from our opening text. As we aim to show, the ability of readers to interpret 
viewpoints being developed within texts is often far from assured, even among quite advanced 
learners. We propose to describe reading problems that we have identified among students at the 
University of Hong Kong (HKU), to seek to account for these problems, and to suggest how teachers 
may make learners more aware of difficulties and misunderstandings that can easily arise in 
interpreting academic writing. 

We do not underrate the difficulties that teachers and researchers face in identifying what a 
reader's interpretation of a text may be, or in deciding when such interpretations can justifiably be 
said to constitute a misreading and not simply an alternative reading of a text (see editorial 
comments in Aiderson and Urquhart, 1984). Readers are invited to consider these issues within the 
contexts of the examples to be presented in this paper. 

We accept that effective reading is interactive, in the sense of Carrel! (1988): successful 
interpretation of meaning in texts will involve the interaction of top-down and bottom-up processes, 
so that a reader's interpretation is not driven solely by either an initially chosen schema or the 
parsing of successive word groupings. We would simply add that a good deal of less effective 
reading is also interactive in nature, as we shall exemplify. 

In seeking to identify and describe students' problems in reading academic discourse, we 
shall draw upon action research involving individual learners as well as work in the experimental 
tradition that compares responses over groups of learners. Such a range of approaches seems to 
us to be appropriate to the purpose.^ 

Our examples in Part One are drawn from teacher comparison of a source text with a 
student's written response, supplemented by discussion with the student of how certain features of 
the source text are to be understood. Such intervention by the teacher is characteristic of action 
research (and teaching), but it inevitably raises the issue of how far a student's introspections, as 
elicited and reported during discussions, may reveal actual problems and processes in reading. We 
would not expect all difficulties to be identified in such ways, but we are confident that the kind of 
interviews we shall describe can bring to light, and help to rectify, some persistent learner 
misunderstandings of words, phrases, grammatical choices or discourse moves in texts. 
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In Part Two, we report on a study in which a short comprehension test was used to elicit 
reader responses in an area where comprehension difficulties were anticipated, and to compare 
tendencies over groups of readers under contrasting corxiitions. A study of this kind raises its own 
questions of application to actual reading. There is always a danger that outcomes may be an 
experimental artefact rather than a genuine elicitation of ways in which readers (mis) interpret 
meanings in texts. Provided, though, that care is taken in interpreting findings, work in the 
experimental tradition will continue to offer useful insights into responses that are widespread in a 
given population of readers, and into factors that can influence such responses. 



Part One: Individual Case-Studies 

We may expect some r- . ''ng problems to be widespread in a class, so that addressing 
these problems in teaching can help learners who have difficulties in common. Yet it is ultimately 
individuals, rather than groups, who have reading problems. One needs a better understanding of 
what individual difficulties actually are than a simple indication of who gives a wrong answer on 
some comprehension test item. 

Individual learning experiences and difficulties can be explored through studies of 
introspective and retrospective data (see Faerch and Kasper, 1987). Interviews that elicit readers* 
own explanations of what they have understood from a text can greatly assist our understanding of 
how learners interpret what they read. A record of such interviews, in the form of transcribed audio 
recordings or of notes written in a teacher-researcher's diary, reveals important information about 
what a learner has understood from a text, the reasons the learner gives for this interpretation, and 
ways in which interpretations of particular linguistic patterns may ( ontribute to or may be influenced 
by an overall (mis)understanding of the text. This is not to claim that researchers' accounts of such 
data are infallibly accurate, but simply that they are based on a fuller and more intensive individual 
study than is the case for most teacher and researcher judgements. 

In relatively privileged teaching situations, it becomes possible to pursue such action 
research when working with individual learners. The following accounts are drawn from work with 
students who were taught individually for some time by one of the authors, K.S. Ip, at the Language 
Centre at HKU. The Centre offers a limited service to individual students who seek to improve their 
English language skills. A student is assigned by the Centre to meet a tutor on a one to one basis, 
and problems are discussed before a 'diagnosis* and advice for further action can be given. 
Students may occasionally return a number of times, depending on the nature of the problem, the 
motivation of the learner and the availability of a tutor. The more sustained study reported here 
involved many additional voluntary hours on the part of the staff member. 

This service to undergraduate students was originally termed the 'Writing Clinic', which 
indicates that problems have most commonly been perceived by teachers and students alike as 
pertaining to writing rather than to other skills such as reading. Although students can now seek 
advice under any of the four skills, few if any 'complaints', to continue with the medical metaphor, 
are identified as 'reading problems' by students. One reason may be that students usually only 
become alerted to their own problems with (written) English when they receive negative feedback on 
tutorial essays. 

One difficulty in presenting our findings is that extended contexts are qften needed for the 
nature of a problem to be fully appreciated, yet a range of illustrations is also desirable. To reconcile 
these demands without making the paper unduly lengthy, we shall report two examples from one 
case study in some detail and others in summary form. 
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The subject of the more detailed case study is a third year arts student reading five papers in 
English literature and linguistics, and three papers in Chinese studies. This particular student is 
chosen for our discussion because of her readiness to engage in introspection. Her high proficiency 
in the first language seems to have given her an awareness of how language works. But perhaps of 
more interest to the discussion here is that she is a highly motivated learner, who makes sure she 
understands the vocabulary, but is not word-bound and uses an interactive strategy to work out the 
meaning of a text. Yet, as shown in the discussion below, she has not been successful. 

This student undertook to read the letters to the editor" column of the South China Morning 
Post, with a view to writing a response to a letter of her choice, in a similar expository style. This 
exercise was negotiated between the tutor and the student: it was felt that the student would show 
more interest in a topic if it is of her own choosing, that she would be exposed to a variety of views 
by different correspondents, and that the exercise would familiarize her with the forms of 
argumentation and refutation. 

The first two examples, discussed in depth, will illustrate how an integrative strategy can still 
result in misreading. 

Example 1 

A letter (SCMP 23/7/90; full text in Appendix A) headed "Discuss this matter openly" on the 
subject of homosexuality includes the following four sentences in the course of an 11 -sentence text, 
starting with: 

a. "While listening to the debate leading up to the recent decision to abolish the law on 
homosexuality, I got the impression that most people see being homosexually orientated as 
an abnormal, hopeless condition which only deserves polite words of consolation." 

The third sentence in the text reads: 

b. "It is ludicrous to think that a homosexual ever chose to be attracted to his own sex." 
The concluding paragraphs include: 

c. "...and therefore we need an openness to discussing homosexuality which is brave and 
patient yet does not compromise moral standards. ..." 

and end with: 

d. "To homosexuals still struggling with their feelings, let's continue to work towards better 
things." 

The student's interpretation of these four points was respectively: 

a. The author sat through the debate, and learnt that being homosexually orientated is an 
abnormal, hopeless condition. 

b. It is ludicrous for a homosexual to think that he ever chose to be attracted to his own sex. 

c. Homosexuals are brave, however, to discuss things in the open. 

d. They are still struggling... 

It is interesting to note the internal consistency of the misinterpretation. A schema was 
tentatively set in (a) as information of common knowledge, that homosexuals are abnormal. This 
was confirmed in (b), that homosexuals are 'ludicrous', a word which the student checked in the 
dictionary. When the student came to (c), she admitted initially suspecting a possible discrepancy 
(though for the wrong reason): why does the author criticize homosexuals and then compliment 
them on their bravery? Her interactive strategy might have required her to rework what the author's 
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view was at this point. However, the concluding sentence (d) that they are still struggling confirmed 
her understanding that they are brave. Hence, her integrated comprehension was that the author, 
despite his disapproval of homosexuals, acknowledges their brave struggle, and asks the public co 
help them. 

University students usually have to handle texts longer than this, which leads to the next 
question: how far into the text will this kind of conscientious but unsuccessful interactive reading 
carry the student before she is too perplexed? Our second example will illustrate the possible extent 
of such misreading. 

Example 2 

A letter entitled "Smoking ban on planes unnecessary* appeared in the South China Morning 
Post (SCMP 31/8/90). This letter is twice as long as our previous example, and carries relatively 
longer sentences. The opening section of the letter is as follows: 

"In recent weeks there have been several letters about smoking on 
aircraft, and this month an article has appeared in part suggesting 
that non-smokers may be at real risk from exposure to passengers 
who happen to be enjoying a cigarette. 

With the World Health Organisation and the British Medical 
Association calling for a total ban on aidine smoking, it is high time 
that a few basic facts were aired on a subject which has become 
contentious for no good reason. 

Whatever anti-smoking activists may say, there simply is no 
reputable scientific consensus which can support so-called "passive 
smoking" as a serious health hazard." 

It is clear to the experienced reader that the author is critical of previous arguments and 
challenges supporters to provide scientific evidence. 

The student felt, however, that the author's position was that "non-smokers may be at real 
risk". Based on this she appeared to have established an initial schema, which she first had to test 
against "^it is high time that a few basic facts were aired on a subject which has become 
contentious tor no good reason". She looked up "contentious" and decided the meaning fitted in 
with the initial schema that 'smokers being at risk is a fact that warrants no controversy*. This 
(mis)interpretation was further fortified by the next few paragraphs, which were read by the student 
in the same light. 

The experienced reader's understanding of the full text (in Appendix B) would be that the 
author refutes a previous correspondent's objections to smoking, and puts fooA^ard the argument 
that smokers have rights too, especially in view of: 

a. the declared absence of statistically significant scientific proof of the risks of passive 
smoking; 

b. the ease of improving air flow in aircraft. 

The student, however, read "Other people's tobacoo smoke may well be a nuisance or an 
irritation for some, in certain circumstances, but... " (the author goes on to explain the logistics) to 
mean 'smoke is indeed a nuisance', which was taken to be additional information, consistent with her 
understanding of the author's meaning. This understanding was further expanded to incorporate a 
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concessive opening "It is true that such measures would cost the airlines money, but..." to mean 
'smokers are troublesome people and to accommodate them would Incur additional costs'. Taken 
together with the logistics in improving the air-flow, which she, as a conscientious reader, does 
acknowledge, her interpretation prompted her to write: 

The (previous) correspondent himself admitted that other people's tobacco smoking is a 
nuisance. This should not be doubted. How can he expect that the air conditioning system 
In such enclosed space to be efficient in eliminating smoke? All the arguments about 
improving the system and costs involved can be eliminated if people don't smoke." 

By this point, the student was well into the text. 

From the above retrospective analysis of the written summary and introspective analysis of 
the persisting interpretation, it is clear that conscientious interactive reading does not always ensure 
comprehension of an author's position. The next question would be: what type of authorial stance is 
most prone to misinterpretation? 

On the basis of instances such as those reported here, it appears to us that students have 
persistent problems with missing the non-supportive stance that an author may adopt towards views 
that he or she is presenting. These instances include cases where an author is critical of another 
view, referring to it only in order to dismiss it; or where the author concedes the possible validity of a 
view, but seeks to draw the reader's attention to other aspects which are presented as more 
important. Whatever the variation, it seems that our students are not sensitized to the intended 
emphasis or weight put on each piece of information. Our next examples (3a, 3b, 3c) illustrate such 
tendencies. 

Example 3a 

This example is taken from another letter headed 'Tam case shows checks and balances 
needed for HK" (SCMP 7/8/90): 

"Of course, no one is quite sure what is going to happen after 1997, 
but because the present regime in China has such a poor record in 
dealing with its own people, and it is well known that China's leaders 
are much displeased with the people of Hong Kong, it has been 
assumed - and wisely so - that precautions need to be taken." 

The text was chosen by another student attending the Writing Clinic: a second-year female 
undergraduate from the Faculty of Arts, reading three papers in English. When asked to identify the 
main point in the written sentence, she felt that every piece of information was of equal importance, 
and they were not necessarily connected: 

1. No one is sure what happens after 1997. 

2. China has had a poor record. 

3. China does not like HK people. 

4. Precautions must be taken. 

It is worth noting here that the student seemed to have been thrown off by the opening: 
"Of course, no one is quite sure..., but..." 
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In our tutorial session, the student's attention was drawn to the two parts of the compound 
sentence connected by the word "but\ An experienced reader would see the author as wanting to 
place emphasis on the second part, which would be the author's own position, and as playing down 
the first pari which serves the function of acknowledging a differing view. The student, however, 
appears to have seen the two parts of the sentence as bearing more or less equal information load, 
with the connection behaving more like an '*and^ as in "I like oranges but I like apples too". 

Comparable difficulties have already been noted in the case study of the third-year student. 

with 

"It is true that such measures would cost the airlines money, but 
"While..., I got the impression that most people see homosexuality as..." 

One may speculate on whether such indicators may loom more prominently and 'authoritatively' for 
readers of certain cultural backgrounds. In all three instances, students had missed the critical 
attitude that follows immediately. 

In another example, it proved difficult tor the third-year student to detect a negative, in this 
case iroriic, authorial view in the sentence: 

Example 3b 

'The Bill of Rights was finally passed at the Legislative Council on 
June 27. The strong opposition based on political 
considerations has only served to strengthen public support for 
the Bill." (SCMP 3/8/90: "Best guarantee for a smooth transition") 

The student's interpretation was. The strong opposition has managed to serve just one function, that 
of strengthening public support'; the student missed the author's ironic indication that the actual 
effect of the strong opposition was opposite to what was intended. 

The earlier letters on homosexuality and on smoking both begin by referring to an existing 
view that their authors would then go on to reject. Another example of such a move appears in a 
letter on the condition of Filipino maids in Hong Kong. The first sentence of that letter reads: 

Example 3c 

"The Commissioner of Labour's reply concerning foreign domestic 
workers spells frustration to those who had hoped for a genuine 
solution.** (SCMP 3/10/90) 

The stance of the letter writer towards the Commissioner of Labour's reply is clearly critical: 
the Commissioner's reply, it is said, only offers frustration to anyone hoping for a real solution. We 
can anticipate that the author is likely to continue by outlining what a genuine solution would 
comprise, and/or how the Commissioner of Labour's reply fails to offer this. The second-year 
student's understanding, however, was The Commissioner of Labour said that foreign domestic 
workers have a lot of frustration'. 

This particular example might indeed be one of 'careless* reading (in which the significance 
of "spells frustration to", as distinct for Instance from opells out the frustration of, was simply not 
attended to), yet it is striking that such misreadings should also persist in obviously 'careful' readers, 
who take time and trouble to look up the meanings of words and to articulate the grounds for their 
own interpretation of a text. 
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It seems that our students tend to miss the critical comment in the author's point of view, 
mistaking it to be a supportive statement. Having formed a tentative but faulty top-down schema, 
'careful' but ineffective readers continued to incorporate new information from the text into the 
schema, and to interpret text by reference to it. Researchers have begun to investigate the nature 
and possible origins of such dominant content as well as rhetorical schemata: see, e.g., Steffenson 
(1986). Further studies into cross-cultural discourse with subjects who are articulate and proficient in 
at least one language may well yield interesting results. 

Part Two: A Quasi-Experimental Study 

Persistent misreading of an author's preparatory moves towards establishing a critical view 
will undoubtedly be serious when students set out to interpret academic discourse. Although our 
earlier examples from readers' letters serve to show that moves of this nature are not peculiar to 
academic writing, they are certainly of importance to it. The extent to which such moves may cause 
problems to student readers cleariy deserves study. 

Work in the experimental tradition can help to establish how widespread certain errors in 
interpretation will prove to be among a population of readers. Comparison between control and 
experimental groups of learners can also indicate the contribution of a particular variable to the 
incidence of such problems, in spite of the variation to be expected between individuals on the same 
task. 

In the above individual examples, we saw that subjects had problems in identifying contrasts 
between a view being reported in a text and the critical or distanced standpoint of the writer 
reporting it. Earlier experimental studies by Allison (1986; 1991) had investigated comparable 
reading difficulties among science students, at the University of Botswana (UB), in cases where 
textbooks presented theories that the textbook writer then went on to show to have been refuted by 
later scientific discoveries. 

This discourse relationship has elsewhere been termed *HypotheticaI-Rear (Hoey, 1983; 
critical discussion in Allison, 7997; 379-80). The basic problem was a widespread tendency among 
readers to take 'Hypothetical' reported theories as correct, despite the subsequent evidence and 
'Real' verdict against them in the text. Comparable problems can be anticipated when an author's 
viewpoint, as inferred by experienced readers, is based on a different line of argumentation or a 
different value system to reported views that are rejected in the text. 

Indications of such reading problems among students at UB had first come from inspection 
of students' own written summaries of a text; subsequently, experimental work elicited reader 
responses to test items focusing on the point at issue. 

Aims of the study 

A study was undertaken by Allison at HKU to document the incidence, in this population, of 
readers' problems in distinguishing an author's disagreement with views being cited (i.e. in identifying 
'Hypothetical-Real' relations) in an instance of academic discourse. 

A further aim was to compare outcomes for two sets of instructions, designed to elicit 
contrasting (selective versus holistic) reading strategies during the task. It was anticipated that 
readers who were encouraged to study the whole text before answering would tend to score higher 
than readers attempting to scan through the text to locate answers. For procedural reasons (see 
Methods section: "limitations"), this prediction was not treated as a formal hypothesis. 
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Methods 

A quasi-experimental study was designed in the summer of 1990, during the writing of one 
unit of a new 60-hour in-sessional course in English for Social Science students. This course 
material was piloted in 1990-91 with students on the Bachelor in Business Administration (BBA) 
programme: the relevant unit was completed in October 1990. 

Students 

The students taking part in the study were first-year students on the BBA programme at 
HKU. These students had just completed an intensive 24-hour induction programme which had 
included introductory work on strategies for reading academic texts. They were in the early stages 
of a 60-hour in-sessional course designed and taught by the Language Centre and one staff member 
from the Department of English. Students were taught in six groups, with group size ranging from 
13-18. This study was integrated into the teaching programme. It involved a total of 77 students: 40 
received the 'selective reading' instructions, and 37 were given the 'holistic reading' instructions. 

It should be noted that these HKU students had reached a higher educational level than the 
pre-entry students in the UB study: they had completed seven as opposed to five years of secondary 
schooling. On the other hand, most secondary schools in Botswana effectively used English as the 
medium of instruction. Reportedly, there were only occasional classes in which the national 
language, Setswana, was used alongside English. The ESL experience of HKU students is likely to 
have been more variable with respect to the use of English or of Cantonese during secondary school 
classes. 

The co-operation of students and teachers was essential to the success of this study and is 
gratefully acknowledged. 

Text and task 

Students had to read a 959 word extract drawn from an introductory sociological text 
(Giddens, 1986: 9-13). This text was the second reading source for the unit, so the students had 
already been introduced to some relevant ideas about sociology as an intellectual enterprise. All 
readers had to respond either 'Yes' or 'No' to each of five test items. There were two different sets 
of instructions for the activity, in accordance with the experimental variable for the study. (The 
extract from Giddens (1986) and the alternative sets of instructions are reproduced in appendix C.) 

The short test asks readers to decide whether the author would agree with each of five 
statements. The items of principal interest for the study are 1 and 2. These test items restate views 
that the author, Giddens, attributes to others (what Comte "claimed" and "believed", or what 
Durkheim "proposed"), but which he himself later explicitly rejects ("... although this type of 
standpoint has been very pervasive in sociology, it is one I reject"). 

Test items 3 and 4, statements actually made by the author, are distractors. Item 5 
resembles items 1 and 2 in being a 'Hypothetical' viewpoint that the author rejects, but, for reasons 
of actual wording, this item would seem less liable to be misinterpreted. This is because the 
statement matches fairly closely the wording used when the author's own 'Real' viewpoint is 
presented in the text. 

Alternative sets of instructions were designed in order to realize the experimental variable 
with regard to encouragement of particular reading strategies, respectively 'selective' (Group 1) and 
'holistic' (Group 2). Thus, the 'Group 1' instructions call on readers to refer to the passage in order 
to answer the items, whereas the 'Group 2' instructions tell readers to read the whole passage first 
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before answering the questions. In each class, these instructions were handed out alternately, and 
readers were urged to follow carefully the instructions they had individually been given. 

It has been noted that Giddens' own reporting of the 'Hypotheticar views that feature as test 
items 1 and 2 precedes his 'Real' rejection of these views in the text. It seems plausible in such 
cases that selective reading by inexperienced or unwary readers might be specially likely to elicit the 
unwarranted inference that reporting a viewpoint also entails endorsing that viewpoint. If this were 
so, the 'selective' readers (Group 1) can be expected, on average, to perform less well on their task 
than the 'holistic' readers (Group 2). This expectation will be explored in the analysis and discussion 
of data. 

The activity was not a special test session for students but an exercise within the teaching 
programme. Students handed in their answers for purposes of the research, but kept copies for pair 
and then whole class discussion. Answers were compared in class immediately after completion of 
the activity. The teachers commented as needed on the ways in which the author's own viewpoint 
was signalled in the extract. 

Limitations 

The term 'quasi-experimental' has been used to emphasize the exploratory nature of this 
study, which is to be seen as an instance of action research using some experimental techniques. 
There had been no piloting of the text used, and no previous studies with the reader population 
being sampled. A full experimental design would have been premature and has not been realized. 
This is the reason that the study includes no formal testing of a hypothesis. 

Specifically, there are two notable limitations in the design. First, the test that was used 
comprises only five items, three relating to the problem of reported views and two serving as 
distractor items. Second, the experimental variable - selective response versus holistic reading of 
the text - was only operationaiized by means of written and spoken instructions to students, A more 
rigorous experiment would have had to present materials differently, possibly asking the 'holistic' 
readers to read the text and summarize its main points before test questions were distributed at all. 
Besides constraints in preparation time, these design features partly reflect the pedagogic role of the 
activity in class, which did not require elaborate or lengthy tasks. 

There was also some variation in actual spoken instructions. Some teachers just 
emphasized the need to read the written instructions, while others specified the nature of the two 
sets of instructions - thus making explicit the alternative reading approaches that these instructions 
were attempting to elicit. This was not considered a serious additional problem, with any effect 
being balanced across Groups 1 and 2, but it would also need to be controlled for in full 
experimental work. 

Statistical treatment 

Item facility values for all items will be shown, and descriptive statistics will present scores 
obtained by readers in the study. I.F. values for items 1 and 2 are relevant to the question of 
incidence of these readers' problems in recognizing that an author disagrees with views that he cites. 

Regarding the additional aim of comparing outcomes for the two treatments, i.e. for the 
'holistic' and 'selective' sets of instructions, descriptive statistics for the two groups in the study will 
be followed by an indication of trends that can be observed. This will be supported by chi-square 
analysis of each item, using two-tailed tests for statistical significance. This procedure was preferred 
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to comparison of group means, as there are only five test items, only two of which are directly 
applicable to the purpose. Item content was judgad sufficiently distinct to support the assumption 
that the test items were independent. 

It is again emphasized that formal hypothesis-testing is not i^eing undertaken in this 
quasi-experimental study: the purpose of the statistical testing is to support the exploration of the 
data obtained and the interpretation of any trends observed. 

Results 

Table 1 shows item facility values for the whole sample, and also for each of the two groups 
constituted with reference to the two sets of reading instructions. 



Table 1 : Item facility values in a reading test calling for recognition of author's viewpoint. 



Item 


Total 


I.F. 


Group 1 


Gp.1 


Group 2 


Gp.2 


no. 


correct 


value 


correct 


I.F. 


correct 


I.F. 


1. 


(i) 
44 


.57 


(ii) 
18 


.45 


(iii) 
26 


.70 


2. 


52 


.68 


24 


.60 


28 


.76 


3. 


53 


.69 


30 


.75 


23 


.62 


4. 


62 


.80 


31 


.78 


31 


.84 


5. 


68 


.88 


33 


.82 


35 


.94 



(i) 77 students took the test. 

(ii) Group 1: 'Selective' reading instructions. 

(iii) Group 2: 'Holistic' reading instructions. 



Table 2 shows the scores obtained by learners under each of the two conditions (selective 
versus holistic reading instructions). 



Table 2: Distribution of scores (out of 5) obtained in 'selective* and 'holistic' groups, 
on a reading test calling for recognition of author's viewpoint. 



Group 1 ('selective') 


Group 2 ('holistic') 


Score 


Number of cases 


Score 


Number of cases 


5 


7 


5 


13 


4 


11 


4 


11 


3 


15 


3 


7 


2 


7 


2 


6 


1 


0 


1 


0 


0 


0 


0 


0 
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Each test item offered learners a binary choice of yes and no. One question is whether 
observed total scores for correct answers significantly exceed the value to be expected on the basis 
of chance (an I.F. of .50). For 'total correct' in Table 1, an I.F. value of at least .62 (representing at 
least 48 correct responses out of 77) is needed for the result to be statistically significant at .05 level 
(chi-square test, two-tailed, one degree of freedom) and thus to indicate that performance was better 
than chance. Overall results show that performance significantly exceeded chance on four items, 
but failed to do so on item 1. Item 1 also shows the greatest difference in results when Groups 1 
and 2 are compared, with the Group 1 success rate being below 50%. 

From Table 2, the distribution of scores in Group 2 is markedly non-normal and reflects high 
success rates for this group on the test, whereas that in Group 1 is closer to a normal distribution. 
Comparison of I.F. values in Table 1 shows higher values for Group 2 in four cases out of five, 
including the two Hypothetical-Real items (1 and 2), for which the differences in I.F. values are most 
marked (differences of .25 for item 1 and .16 for item 2). 

Chi-square values for each item were calculated. For item 1, the difference in distribution of 
results over Groups 1 and 2 yielded a significant chi-square value of 4.033 (p<.05, Id.f., 2-tailed). 
Item 5 was not tested as expected values fell below 5. Other values were not significant. Results 
indicate that item 1 was the most difficult of the five, and also that it was significantly more difficult 
for the group of learners who were instructed to read the text selectively in order to answer the 
items. 



Discussion of Results 

Sufficient evidence was obtained to support the view that problems in identifying an author's 
viewpoint (in the case of Hypothetical - Real relations) will be Tound quite extensively among readers 
from this population, and will merit further study as well as pedagogic attention. 

There was some indication that a selective answering strategy may exacerbate the incidence 
of such problems. The use of chi-square to assess differences in distribution of answers over each 
test item can be criticized, because repeated use of a measure affects 'significance' of outcomes. In 
this case, however, there were only two items for which an effect was anticipated because the other 
three items served rather as controls. Furthermore, only a two-tailed significance test was used. It 
seems reasonable to accept that the trend observed on item 1 can be attributed to something other 
than chance. It appears likely that for this test item the higher rate of success among readers given 
'holistic' instructions was partially a result of the reading strategy being encouraged in that group. 
Obviously, this issue would need more rigorous and extensive study, involving other learners, texts 
and tasks, before firm general conclusions could be offered. 

One more limitation was suggested by other findings after the study. A mid-course 
questionnaire revealed that the teaching unit in question received relatively low student ratings for 
utility and especially for interest. This need not imply that the particular activity we have described 
was so rated; teacher observation had suggested that the test and immediate feedback successfully 
focused learner attention on both the task and subsequent discussion of how and why the writer had 
disclaimed some of the views he cited. Nor would general doubts about level of interest explain why 
performance should be less successful on a particular test item or under one of two sets of 
Instructions for the task. None the less, we would like to emphasize our earlier warning that results 
on one reading test may not fully reflect reader performance in other situations. It is for reasons 
such as this, and not merely as a polite convention, that the usual call for further research is 
warranted. 
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Concluding Remarks 

Moving from informal impressions of possible reader problems into carefully documented 
research is important, but is not easy. We are fully conscious that our respective studies have 
involved a small number of individual subjects or a small number of reading test items, and that 
more comprehensive research is needed to provide a stronger basis for generalizations. Yet we 
need not be unduly apologetic. Ours is a field in which generalizations have to be made, by 
teachers and by administrators, and in which authoritative research in the particular learning context 
is usually postponed to some ideal future. If generalizations about students' learning problems can 
become somewhat better informed - and if the remaining complexities and unresolved questions are 
made that much more apparent to all concerned - we believe that it is possible for findings from 
studies such as our own to be put to sensible use. 

In the present case, we have provided reasons to suggest that students' reading problems 
are more serious and persistent than might have been anticipated, and, in particular, that readers' 
difficulties in identifying the critical stance of authors towards viewpoints being reported concessively 
or contrastively could usefully be addressed by teachers of English for academic purposes. For this, 
we would emphasize the value of individual face-to-face discussions in which students are iriduced 
to articulate their interpretations of texts and linguistic grounds for reaching these interpretations. 
We would also argue in favour of whole-text or extended extract rather than piecemeal search 
reading when the aim is to determine authorial viewpoint. 



NOTES 

1. The recent literature on classroom language learning research encourages exploratory and 
qualitative studies in order to generate insights, and to throw light on particular situations and 
experiences, without too much concern for immediately demonstrable generalizability of findings: 
see, e.g., van Lier (1983). This cannot mean that workers in our field no longer regard 
hypothesis-testing and generalizability as important, but it indicates that we have learned how difficult 
it is even for experimental research to furnish conclusions that are valid beyond particular situations. 
The need for comprehensive and rigorous study before broad claims are made has to be respected 
if applied linguistics is to remain a responsible mode of enquiry. See also Chaudron (1988). 
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APPENDICES 
Appendix A 

Letter in South China Morning Post (23/7/1 990) (reproduced with permission). 

Discuss this matter openly 

While listening to the debate leading up to the recent decision to abolish the law on 
homosexuality, I got the impression that most people see being homosexually-orientated as an 
abnormal, hopeless condition which only deserves polite words of consolation. 

They fail to realise that a homosexually-orientated person goes through a very human and 
not uncommon struggle. It is ludicrous to think that a homosexual ever chose to be attracted to his 
own sex. 

A crucial distinction must be made between sexually-active homosexuals and people with a 
homosexual orientation who have not entered into a homosexual relationship. The first group has 
made a voluntary decision to lead a homosexual lifestyle, while the second still desires to deal with 
the problem without giving in to his desires. 

A homosexually-orientated person will go through a difficult period of confusion and 
alienation while trying to define his identity. Acceptance and encouragement from family and friends 
are paramount in keeping him from resorting to homosexual relationships to obtain the love and the 
acceptance lie needs. 

Homosexuals are no more at fault for their feelings than heterosexuals are for being 
tempted. How we choose to deal with the temptations is another matter, and therefore we need an 
openness to discussing homosexuality which is brave and patient yet does not compromise moral 
standards. Relying on laws or brushing off the problem as a repugnant disease requiring clinical 
therapy will definitely not protect Hongkong from what it fears the most, wide-spread homosexual 
activity. 

To homosexuals still struggling with their feelings, lef s continue to work towards better 

things. 

RAYMOND CHIU 
University of Toronto 
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Appendix B 



Letter in South China Morning Post (31/8/1990)(reproduced with permission). 

Smoking ban on planes unnecessary 

In recent weeks there have been several letters about smoking on aircraft, and this month an 
article has appeared in part suggesting that non-smokers may be at real risk from exposure to 
passengers who happen to be enjoying a cigarette. 

With the World Health Organisation and the British Medical Association calling for a total ban 
on airline smoking, it is high time that a few basic facts were aired on a subject which has become 
contentious for no good reason. 

Whatever anti-smoking activists may say, there simply is no reputable scientific consensus 
which can support so-called "passive smoking" as a serious health hazard. 

All the comment is based upon opinion which deplorably has been presented as fact, now 
believed as gospel by many smokers and non-smokers alike. This is especially unfortunate as new 
research is emerging which is illustrating how flawed and uncertain some of the earlier work has 
been. 

Other people's tobacco smoke may well be a nuisanc or an irritation for some, in certain 
circumstances, but on aeroplanes in particular, the interior airt' - system and the change of cabin 
air content, every few minutes, militates against any significant problem. 

Moreover if airlines chose to increase the cabin airflow rates, which would be no bad thing 
for a number of reasons, together with raising what is often a most uncomfortably low level of 
humidity, the opportunity for complaint by non-smokers would disappear completely. 

It is true that such measures would cost the airlines money, but the increased costs would 
be extremely small, such as adding less than HK$5 to the price of a seat from Hongkong to 
Singapore. 

Interestingly, flight deck crews enjoy a cubic foot per minute rate of ventilation around the 
150 mark, whereas a passenger norm is seven. 

An accepted comfort level should be no less than 20 cfm. And when humidity ought to be 
at 40 per cent, on long haul flights, passengers commonly experience as little as two per cent. So 
airlines have some responsibility to make a few housekeeping improvements themselves, including 
better screening from fuel vapours and ozone, instead of taking the all too easy option of blaming 
the visible tobacco smoke for those dry eyes, sore throats and the general lassitude so familiar to 
regular air travellers. Research has revealed that tobacco smoke in aeroplanes has next to nothing 
to do with such symptoms. 

Even the chance of cancer from cosmic radiation while flying has been held to offer a higher 
statistical risk than health problems from airline smoking, but what do we ever hear about cosmic 
radiation, or being somehow screened from that? You have a predicted death rate from this 
radiation being discounted as minor, and of no genuine concern in terms of statistical risk, while the 
practically negligible alleged statistical risk from "passive smoking" is cited as a reason for a 
worldwide ban! 
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Finally, have the airlines fairiy asked their passengers, a large minority of whom would like a 
facility to smoke on board, whether these bans are either welcome or necessary? 

The level of complaint about smoking on aircraft is in truth extremely low, and an example 
fronn the US makes the point. Over a 10 year period there, only a single smoking complaint was 
received for every one million passenger miles flown. 

So just where is the groundswell of opinion pushing the airlines into ignoring the personal 
freedoms of people who wish to take pleasure from a relaxing smoke while flying? 

A.D.C. TURNER 
Managing Director 
Asian Tobacco Council 
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Extract from Giddens (1986) and two task sheets. 

READING (2). SOCIOLOGY: A Definition 

... In the light of these remarks, a definition can be offered of the subject as follows. 
Sociology is a social science, having as its main focus the study of the social 
institutions brought into being by the industrial transformations of the past two or 
three centuries... 

...Many of the prominent thinkers associated with the development of sociology were 
impressed with the importance of science and technology in contributing to the 
changes they witnessed. In setting out the aims of sociology, therefore, they sought 
to duplicate, in the study of human social affairs, the successes of the natural 
sciences in explaining the material world. Sociology was to be a 'natural science of 
society'. Auguste Comte, who lived from 1798 to 1857, and who coined the term 
'sociology', gave this view its clearest and most comprehensive formulation. All the 
sciences, he claimed, including sociology, share an overall framework of logic and 
method; all seek to uncover universal laws governing the particular phenomena with 
which they deal. If we discover the laws governing human society, Comte believed, 
we will be able to shape our own destiny in much the same way as science has 
allowed us to control events in the natural world. His famous formula, Prevoir pour 
pouvoir (to be able to predict is to be able to control), expresses this idea. 

Since Comte's time, the notion that sociology should be fashioned upon the natural 
sciences has been the dominant view of the subject - although it has certainly not 
gone unchallenged, and has also been expressed in various differing ways. Emile 
Durkheim (1858-1917), one of the most influential figures in the development of 
sociology in the twentieth century, continued some of Comte's emphases. 
Sociology, he declared, is concerned with 'social facts*, which can be approached in 
the same objective way as the facts with which the natural sciences deal. In his 
short, but very influential book The Rules of Sociological Method (1895), Durkheim 
proposed that social phenomena should be treated like things: we should regard 
ourselves ?s though we were objects in nature. Thereby he accentuated the 
similarities between sociology and natural science. 
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As I have mentioned earlier, although this type of standpoint has been very pervasive 
In sociology, it is one I reject. To speak of sociology, and of other subjects like 
anthropology or economics, as 'social sciences', is to stress that they involve the 
systematic study of an empirical subject-matter. The terminology is not confusing so 
long as we see that sociology and other social sciences differ from the natural 
sciences in two essential respects. 



(1) We cannot approach society, Ci 'social facts', as we do objects or events in the 
natural world, because societies only exist in so far as they are created and 
re-created in our own actions as human beings. In social theory, we cannot treat 
human activities as though they were determined by causes in the same way as 
natural events are. We have to grasp what I would caH the double involvement of 
individuals and institutions: we create society at the same time as we are created by 
it. Institutions, I have said, are patterns of social activity reproduced across time and 
space. It is worthwhile reflecting for a moment upon what this involves. To speak of 
the 'reproduction' of social conduct or social systems is to speak of the repetition of 
similar patterns of activity by actors separated from each other in time and space. It 
is very important indeed to stress this point, because much social theory - including 
that of Durkheim - is pervaded by a tendency to think in terms of physical imagery, a 
tendency which can have damaging consequences. Social systems involve patterns 
of relationships among individuals and groups. Many sociologists picture these 
patterns as rather like the walls of a building, or the skeleton of a body. This is 
misleading because it implies too static or unchanging an image of what societies 
are like: because it does not indicate that the patterning of social systems only exists 
in so far as individuals actively repeat particular forms of conduct from one time and 
place to another. If we were to use this sort of imagery at all, we should have to say 
that social systems are like buildings that are at every moment constantly being 
reconstructed by the very bricks that compose them. 



(2) It follows from this that the practical implications of sociology are not directly 
parallel to the technological uses of science, and cannot be. Atoms cannot get to 
know what scientists say about them, or change their behaviour in the light of that 
knowledge. Human beings can do so. Thus the relation between sociology and its 
'subject-matter' is necessarily different from that involved in the natural sciences. If 
we regard social activity as a mechanical set of events, determined by natural laws, 
we both misunderstand the past and fail to grasp how sociological analysis can help 
influence our possible future. As human beings, we do not just live in history; our 
understanding of history is an integral part of what that history is, what it may 
become. This is why we cannot be content with Comte's idea of Pr4voir pour 
pouvoir, seen as social technology. In the social sciences, we are addressing other 
human beings, not an inert world of objects. It is often precisely by showing that 
what may appear to those involved as inevitable, as unchallengeable - as resembling 
a law of nature - Is, in fact, an historical product, that sociological analysis can play 
an emancipatory role in human society. At the same time, sociological analysis 
teaches sobriety. For although knowledge may be an important adjunct to power, it 
is not the same as power. And our knowledge of hisiory is always tentative and 
incomplete. 



Source: Giddens, Anthony (1986), Sociology: A brief but critical introduction (second edition). 
London: Macmillan Education. Extracts from pages 9-13. 
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Reading 2: Task sheet. 

Group 1: Instructions. Complete the task individually. Then pair up with a member of Group 2. 

Answer the items below one by one, by referring selectively to the passage. Do not change any 
answers once you have written them. 

As soon as you have finished, make a copy of your answers for your own reference (Just copy your 
answers, not the question items). Then immediately hand in this task sheet. Then read the whole 



Questions. Below are five statements. Does the writer (Giddens) agree with them? Answer YES or 
NO in each case. 

1. Aims and methods of sociology should be modelled upon the natural sciences. 



2. Sociology should treat ^social facts' like things in nature. 

2. Answer: 

3. We can call sociology a 'social science' to emphasise its systematic study of empirical subject 
matter (subject matter that can be observed and measured). 



passage. 



1 . Answer: 



3. Answer: 



4. Social systems involve patterns of relationships among individuals and groups. 



4. Answer: 



5. Practical applications of sociology are like the technological applications of science. 



5. Answer: 



Name of student: 



Remember to make a copy of your answers before you hand in this completed task sheet. 
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Reading 2: Task sheet. 

Group 2: Instructions. Complete the task individually. Then pair up with a member of Group 1. 

Read the whole passage first. Then answer the questions below. Do not change any answers once 
you have written them. 

As soon as you have finished, make a copy of your answers for your own reference (Just copy your 
answers, not the question items). Then immediately hand in this task sheet. 

Questions. Below are five statements. Does the writer (Giddens) agree with them? Answer YES or 
NO in each case. 

1. Aims and methods of sociology should be modelled upon the natural sciences. 

1 . Answer: 

2. Sociology should treat 'social facts' like things in nature. 

2. Answer: 

3. We can call sociology a *social science' to emphasise its systematic study of empirical subject 
matter (subject matter that can be observed and measured). 

3. Answer: 

4. Social systems involve patterns of relationships among individuals and groups. 

4. Answer: 

5. Practical applications of sociology are like the technological applications of science. 

5. Answer: 

Name of student: 

Remember to make a copy of your answers before you hand in this completed task sheet. 
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Introduction 

In recent years, interlanguage studies have been characterised by a concern to formulate 
and test hypotheses of a universal order of second language acquisition (SLA). This exciting and 
exacting preoccupation has led to a de-emphasising of the influence of the native language (NL) on 
the acquisition of a target language (TL). However, as Rutherford (1983) argues, there Is no reason 
to suppose that notions of a universal order of acquisition exclude consideration of the proposal that 
TL acquisition may be shaped, coloured and accented in varying degrees by the transfer of 
typological features from NL. 

This realisation has prompted investigation into a wide range of crosslinguistic influences. 
Gundel and Tarone's (1983) psycholinguistic exploration of the acquisition of a feature of English 
discourse cohesion (pronominal anaphora) by Chinese and Spanish students is illustrative of this 
trend. Odiin's (1989) enquiry, sociolinguistic in orientation, evaluates the evidence for the transfer of 
politeness conventions in the interpersonal communication of German learners of English. Trevise's 
(1986) examination of the evidence supporting the notion that French is in the process of becoming 
topic-prominent in terms of an information-or'snted ordering of syntactic constituents is entirely 
relevant to the concerns of this paper. 

It is worth noting that investigations into this kind of transfer complement rather than attempt 
to invalidate or replace a more traditional, parts-focussed contrastive analysis. Wong (1988) has 
argued strongly that, at sentence level, many examples of language production perceived as 
demonstrating typological influence many be accounted for in terms of incomplete mastery of TL 
morpho-syntactic systems. Two aspects of the same acquisition problem are perhaps in evidence 
here. The native speakers of a topic-prominent (t-p) language, and indeed t-p languages themselves 
tend to have reduced grammatical systems, may be predisposed to transfer this prominence and, at . 
the same time, to have a great deal of work to do in acquiring the grammaticisation demanded by a 
subject-prominent TL. The point surely is that both processes are at work simultaneously and do not 
stand in mutual contradiction or exclusion. 

However, Wong does not deny the possible existence of typological transfer, and one way 
out of this impasse is to investigate the possible influence of fossilised typological nfianifestations on 
the accent of longer stretches of discourse, since at the interface of segmental and pragmatic 
comparisons of NL and TL is the ordering of sentence constituents as information-bearers. The 
present study then is located at that interface: a location which Halliday (1985: xvii) appears to view 
as an essential point of departure for the principled analysis of discourse: 

A text is a semantic unit, not a grammatical one. But meanings are 
realised through wordings; and without a theory of wordings - that Is, 
a grammar - there is no way of making explicit one's interpretation of 
the meaning of a text. 

Halliday (op.cit.r xxii) again: 

A text is meaningful because it is an actualisation of the potential that 
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constitutes the linguistic system; It is for this reason that the study of 
discourse (text linguistics) cannot properly be separated from the study 
of the grammar that lies behind it. 

My main interest is enquiring into topic-prominence in Cantonese interianguage stages of 
English. This is so because word ordering concerns are naturally subsumed under an enquiry into 
the operations of topic. I take it as axiomatic that topic-prominence is conceived in the 
psychological deep structure, that topic bears information which provides a frame of reference for 
the written discourse in which it appears (that is until another topic is selected as the discourse 
develops) and with Thompson (1978) that ordering of syntactic constituents is the surface structure 
realisation of topic-prominence. First however it is necessary to examine Cantonese typdogically for 
topic-prominent surface structure manifestations to adduce evidence to support these initial claims. 



Surface Structure Parameters of Topic-Prominence derived from Spoken Cantonese 

1. Absence of the empty-subject place-holders required to preserve grammatical word order in 
English, "there is" and "it is": 



Very hot! 
It^s hot! 

2. Appearance of "heavy" double subjects where the first (locative or temporal adverbial) 
subject functions as topic in the information structure: 

^ m Pi-ri A g s ti fft n , 

China mainland people living conditions very poor. 
People in mainland China live in very poor conditions. 

3. Topic, not grammatical subject, controls redundant co-referential pronoun deletion: 



i§ ffl m %oi!gori -Mr m m w n -tt ft , 

His sister cool< that fried rice very badly so I not eat. 

His sister prepared the fried rice very badly so I did not eat it. 

4. Topic takes a predicate but this is not necessarily a verb as in English. Thus, "serial" verbs 
occur with no proximal subjects. At certain interianguage stages this lack of subject is 
compensated for by the use of sentence-initial existential "there is/are": 



Pt S^- '. R , 

Have many people want emigrate. 
*There are many people want to emigrate. 
Many people want to emigrate. 
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Generally speaking Chinese verbs are not oriented to the expiication of actor-object relations 
which is, of course, their primary function in English. 

5. Relatively free choice as to the constituent selected as topic. English too tolerates some 
choice but then questions of marked and unmarked language use come into play. This 
point is picked up and amplified later. 

6. The premodification of head noun by constituents which would be embedded in English: 



•±ci? th": rT'-'r. M [H A 15 rr^ ^. ^ 

Live in big cities America people often very poor. 
American people living in big cities are often poor. 

7. Early and prominent (sentence-initial) announcement of topic as a frame of reference for the 
ensuing discourse: 



:g tie Igi ^ 4^ 9 

Boss's office is where? 
Where is the bosses office? 



om 14 » ft T? m a ' . 

That matter I am responsible person. 
I am in charge of that matter. 

8. Marginalisation of tne use of the passive construction. Interlanguage samples which appear 
to be failed attempts to use the passive in English nnay in fact be interpreted as 
topic-comment formulations: 

Topic Comment 
Film in Hong Kong,/can buy very cheap. 
One can buy film in Hong Kong very cheaply. 
*Film in Hong Kong can be bought very cheaply. 

The relatively rare appearance of passives in Chinese is perhaps not surprising. As a 
naturally topic-prominent language it has no need of a transformation which is used to foreground 
the objective information focus of a particular sentence in subject-prominent languages. Zhou and 
Feng (1983) provide evidence for the corollary of this; that only a snnall number of English passive 
constructions may be translated directly into Chinese. 

There is a real need to carry out research in this area of Chinese-English interlanguage, 
since Cantonese learners, as distinct from Mandarin NL subjects, have received little attention in 
terms oftypological analyses of their TL production. Exceptions to this are Leung (1987) and Sung 
(forthcoming, 1991). There is, of course, good reason for the a priori assumption that Cantonese 
learners will produce similar results to Mandarin subjects since the major dialects of Chinese share a 
similar written code and major linguistic systems, but this assumption should not go unanalysed or 
untested, particularly in view of the fact that, in the Hong Kong context, students have to operate in 
Cantonese, written Modern Standard Chinese and English. The interflow of crosslinguistic influences 
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is complex and subtle, and it may be difficult to trace associated transfer features accurately to 
source. The diglossic situation just described with regard to Modern Standard Chinese and 
Cantonese makes it all the more important to emphasise that this paper is primarily concerned with 
Cantonese subjects, and is underpinned by the belief that these subjects frame their written English 
output broadly in accordance with the norms of spoken Cantonese. 

The most severe linguistic trials of the Cantonese learner of English may revolve, then, 
around the central issue of perceiving and successfully producing the intricate subject-predicate 
frameworks needed to carry and develop thematic information in English. However, transfer of 
topic-prominence to English cannot be assumed unless the necessary and sufficient conditions 
discussed below are fulfilled. 

A Theoretical Basis 

1. NL and TL must be similar, to some crucial extentjn the possibilities available for the 
ordering of sentential constituents. If the possibilities for word order are radically different 
then transfer is much less likely to occur. 

2. A feature available for potential transfer is likely to be an unmarked option in the surface 
coding of NL, and is transferable to TL via: 

a) the learner's increasing, but intuitive and tacit, awareness of this NL feature. 

b) the learner's simultaneous lack of awareness of the marked and unmarked option 
possibilities and restrictions this feature possesses in the TL 

These conditions are similar to those proposed by Jordens and Kellerman (1981) and are in 
broad concurrence with Meisel's view (1981) that the clearest-cut evidence for NL influence on TL 
production is to be found in word order, since this phenomenon is so rarely transferred in its 
entirety. 2b is my own formulation. Some elaboration might be helpful in explaining these 
conditions. With regard to 1, for example, native speakers of Japanese, a prototypically SOV 
language, apparently never transfer verb-final ordering to SVO English (Rutherford op. cit). For 
transfer to take place there must be a degree of what might be termed "apparency" through which a 
syntactic feature of NL is felt to be sufficiently close to TL norms as to justify logical mapping. In 
terms of word order, Cantonese is an SVO language, but not as rigidly so as English, since it 
appears to allow pragmatic and information-imparting protocols to predominate over formal syntactic 
concerns. This syntactic flexibility allows discourse topic to be emphasised over grammatical 
subject. A degree of propositional explicitness is present which English muffles under a dense layer 
oi grammaticisation. Hawkins comparing English and German syntax (1980, and quoted in 
Rutherford) had this to say: 

English speakers have, in effect, more work to do in extracting 
meaning from form. They must systematically exclude contextually 
inappropriate interpretations from ambiguous and semantically 
diverse surface structures; they must reconstruct semantic 
argument-predicate relations over often large syntactic domains; and 
they must infer semantically relevant material which is not present in 
surface form. 

This argument recurs as the discussion of the whole subject develops. However, the point 
which needs to be made clear here is that Cantonese and English fulfil the first condition for 
transferability of word order form NL to TL, that of apparent similarity of word order. 
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The second condition relates to the Sl-A stage at which learners become increasingly aware 
of typological features of NL, but not of the possibilities and restrictions that exist for these features 
in TL It is reasonably safe to assume with Sung (op.ctt) that Cantonese, like Mandarin, is located on 
the language typology continuum rather closer to the pragmatic word order (PWO) end. while 
English occupies a position very close to the grammatical word order (GWO) extreme. A PWO 
language is so oriented as to allow the communicational imperatives of imparting information and 
providing clarity of reference to dominate word order norms. In sharp contrast to this, GWO 
languages require fairly rigid adherence to syntactic norms, even to the extent that semantically- 
empty place-holders are used to retain fundamental syntax, no matter what the discoursal and 
pragmatic requirements of the context. 

Markedness 

To this point, I have been referring to unmarked language options. Once marked variants 
are considered (in English, samples not having grammatical subject in thematic sentence-initial 
position) the contrast referred to begins to assume the quality of deceptive apparency that is a 
necessary precondition for transfer. Now, PWO I take to be an unmarked option for word ordering 
in Cantonese and, in concurrence with the second condition above, this ordering is available for 
transfer to TL when and as learners become increasingly aware of it as a basic shaping feature of 
their NL Learners then transfer the feature, more or less overtly or covertly depending on level of 
proficiency, to a GWO TL which either does not tolerate the feature (in which case a gross error is 
cleariy in evidence) or has the feature as a marked option (the manifestation may then be 
grammatically-sound but "odd" or "strong" in tone to the TL native speaker). In the latter case, the 
learner has strayed into marked TL language use, because of the retention of PWO from NL Higher 
proficiency learners seem to utilise a range of devices to accomplish this retention, perhaps 
consciously to avoid jarringly basilectal TL output. The following authentic samples, taken from the 
academic writing of young adult post-intermediate level students studying English as part of a 
programme of vocational training, illustrate this point: 

Device Defined Topic Comment 

1. For/those staff who acted as senior onGs,/they earn more salary. 

Device Defined Topic Comment 

2. According to/our calculations on this matter,/lt shows that we will have lack of time. 



Device Defined Topic 

3. Concerning/our academic report belong to vocational studies,/ 

Comment 

it must be written up carefully and well presented. 

In these examples the eariy and emphatic (marked) appearance of topic is followed by, in 
information structure terms, a comment on it. That the sentences above are conceived in 
topic-comment terms is supported by the precise placement of a comma at the topic-comment 
boundary, and by the undeleted co-referential pronoun which picks up and reinforces the 
intra-sentential influence of the topic. 

Topicalisation then is a psychological entity manifested by the wording chosen to realise its 
primary role in Cantonese, that is as a point of departure or reference for the ensuing discourse. 
Lower proficiency learners by contrast, tend to produce TL language samples which are largely 
stripped of any attempt to mask the topicalising influence of the NL Their TL production, as one 
would expect, is closer to NL norms and, in particular, the exophorically-oriented norms of spoken 
NL 
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Topic Comrmnt 

Hong Kong's complicated traffic problem,/it will need many people to solve. 

It is rarely possible to account for the overall "oddness" of much of the writing produced by 
Cantonese learners in terms of the sum total of low-levei lexicostructural errors present in particular 
vexts. However, taking the point of departure for contrastive analysis at the macrostructural level of 
whole text presents problems too. Kaplan's pioneering (1966) hypothesis of the transfer of 
non-linear rhetorical organisation to a discourse-linear TL by Chinese learners of English takes as its 
point of departure the comparison of the Anglo-American academic essay and the classical Chinese 
composition. Kaplan detected digressive circularity and frequent occurences of tangential viewpoint 
in the English writing of Chinese subjects which he assumed was transferred from NL writing and 
reflected "oriental" patterns of thought. 

More recently Malcolm and Hongjio (1988) have produced empirical data which seems to 
suggest that English native-speaker writers of expository compositions follow the linear process of 
structural development Situation-Problem-Solution-Evaluation, while Chinese writers of English 
in the same genre might sometimes use the NL pattern Thesis-Development of Thesis-Contrast 
to Thesis-Indefinite Conclusion. Despite the obvious interest of this line of enquiry it is not of 
direct relevance to the concerns of the present study. It is also important to make clear that the 
research findings available were all extremely tentative, and that the "oriental" thought patterns 
detected in samples of English writing may in fact be ascribed more accurately to developmental 
rather than transfer influences (Mohan and Lo, 1985). 

To return to matters of direct concern, English does, of course, allow topicalisation to occur 
in marked information structure. As an alternative to unmarked forms such as: 

Subject Predicate 

Tony/likes Smith's ideas on education. 

One might find marked object fronting: 

Topic Comment 

With regard to education, /Tony likes Smith's ideas. 
Or participant as topic: 

Topic Comment 

As tor Tony,/he likes Smith's ideas on education. 

However, such marked constructions lose their emphatic quality when a writer overproduces 
them. For this reason these constructions are comparatively infrequent in native-speaker written 
production. My assumption, again in line with the second condition above, is that Cantonese 
learners become aware of the PWO (topicalising) nature of their NL but not that this constitutes 
restricted marked usage in a fundamentally GWO language like English. 

Elicitation Procedure 

I decided to carry out a small-scale procedure to elicit the production of topicalised 
structures in the English of mature, high proficiency students. I targeted this group since I wanted to 
ensure a relative paucity of the surface lexicogrammatical errors that can make detection of 
transferred discourse phenomena so opaque and indeterminate. 

Using the Oxford Placement (Structure) Test Parts 3B1 and 3B2 (Allan, 1985) I selected the 
fifty highest-scoring students from one hundred and thirty to take part in the elicitation activity. A 
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word is in order about the test instrument in that extensive trlalling in my Institute has shown that 
OPT 3 discriminates far more effectively among Cantonese native-speaker subjects than the other 
structure tests in that series. The students selected all scored 160 or above in the test. This 
correlates to upper-Intermediate to advanced status in terms of international measures of language 
proficiency in English. 

I then set a short writing task entitled "How the changing roles of husbands and wives is 
affecting marriage and the family in Asia"; a propositionally rich and syntactically complex topic. My 
intention was to elicit only a few lines of unmonitored and uncorrected TL production from each 
student, so a time limit of just 10 minutes was imposed. 

Findings and Discussion 

There was no occurrence of structures that could be construed as "putative passives", or of 
incorrect co-referential pronoun deletion, heavy double subjects or extensive premodiflcation of head 
noun. However, there were occurences of very emphatic topic announcement through the overt 
fronting of objective information; the so-called "Ross-topicalisation" (cited in Barry, 1975: 8) and 
subjects did avail themselves of a relative free choice in the selection of the constituent identified as 
topic. Interestingly, the corpus contained nine existential + serial verbs constructions. Most higher 
proficiency learners had clearly passed through the SLA stage at which TL production is 
characterised by gross NL influence, but it does seem that, lacking the empty subject in NL, even 
the subjects under consideration were hypersensitive to this contrasting TL feature and 
overcompensated by using the existential + serial verbs construction unnecessarily. The sample 
below is taken from the corpus: 

There are many young and educated Asian women nowadays want 
to marry but cannot because they lack suitable partners to select. 

The difficulty in attribution of topic-prominence at high levels of proficiency is highlighted by 
this example. It could be claimed that the student has simply failed to supply the subject 
co-referential pronoun \vho" to lead the embedding of the subsequent defining clause; the 
incomplete mastery of TL systems argument. However, I tend to favour the interpretation that the 
sentence was conceived in topic-prominent terms; the extended topic here, as with all topics, 
controls co-referential pronoun deletion. Indeed, the non-appearance of the pronoun many be 
interpreted as a typologically-induced failure to conform to subject-predicate norms; the relative 
pronoun here being needed to fill the grammatical subject slot to take second predicate "want". This 
interpretation considers the observed wording as the surface structure manifestation of language 
conceived primarily in terms of topic-prominence. 

To return to the data, apart from the nine sentences mentioned above, a further twenty-eight 
sentences were analysable as topic-prominent in construction; together these sentences represented 
about 18% of the data. Most in the latter group were, as already indicated, examples of 
"Ross-topicalisation". A number of these are displayed and discussed below: 

Concerning the family, it will benefit from the changing roles. It will be enhanced by 
the husbands' new role of child rearing aid. It is because previously the husband 
gave no aid. 

The device used here to herald the topic, is a direct translation of Chinese 1^ B .A 
topic-prominent language clearly has need of such introductory, defining devices, but in marked 
English information structure "as for ..." or "with regard to..." or "on the subject of..." etc are the 
acceptable forms. Sentence-initial use of "concerning" in a left-dislocated topic in English is nearly 
always used to refer to something known to all participants in the discourse and the whole topic 
construction is often followed by grammatical subject to restore SVO ordering as rapklly as possible: 
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Concerning the trip to Paris, I was wondering if we could cancel it. 

The unmarked word order option would, of course, give sentence-initial prominence to 
grammatical subject with the information focus falling on the object complement. 

I was wondering if we could cancel that trip to Paris. 

The three semantic units available for selection as topic in the title viz. changing roles of 
husbands and wives, marriage, and family were in fact all topicalised in the data. The student below 
chose changing roles to begin the piece: 

Change in the roles of the genders, it has come about due to many 
external factors, it is because nowadays both partners must go out to 
work and therefore have the same responsibilities in the house. 

Not only is the topic announced emphatically, but the topic-related pronoun picks up, 
reinforces and carries the topicalised constituent through the discourse. The second 
(sentence-initial) pronoun may be taken as evidence of the inter-sentential potential of topic. It is 
worth reiterating the fact that this type of topical isation in TL always involves the precise placement 
of a comma at the topic-comment boundary; a surface representation of the deep structure process 
in operation. 

The following example is a very sophisticated one, combining notions of both marriage and 
professional desires in contrast: 

For many people who wish to get married but who also wish to fulfill 
professional ambitions in a wider context, they must develop a wkje 
range of domestic and occupational skills as well as possess unlimited 
energy! 

The sentence stands first in the piece of writing and demonstrates a very impressive degree 
of control and embedding, but all within a "softened" version of the topic-comment framework 
retained from NL A final example of the deep-structure conception of tofiic and its pervasiveness 
even at higher levels of proficiency is given below: 

According to research evidence, it shows **^at more Asian couples 
are getting divorced nowadays. 

The frame of reference "my research" holds an importance and influence over the discourse 
that it does not possess in the native-speaker options displayed below: 

According to my research, more Asian couples are getting divorced 
nowadays. 

or 

More Asian couples are getting divorced nowadays according to my 
research. 

or 

My research shows that more Asian couples are getting divorced 
nowadays. 
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Discourse Accent 

It can be argued, then, that in the examples information is being processed and presented in 
a manner very different from the way a native speaker would carry out these operations. Chinese 
discourse frames seem to be intruding into English syntax and the effect is both powerful and 
puzzling, since often, but by no means always, the TL used is ordered in a syntactically marked way 
and, as such, breaks the given-new contract underlying English infornriation structure. If enough 
marked "contract-breaking" topicalisations are present in the text, the impression conveyed to the 
native-speaker reader of English will be of disjointed and fragmented development of the messages 
embodied in the discourse. 

The quality of staccato "horizontal listing" which pervades a good deal of the written English 
discourse produced by Cantonese learners could be related to toplcalisatlon and the naturally 
co-occuring lack of embedded thematic elements, and concomitant non-appearance of adequate 
cohesive ties. This is not surprising if, in the deep structure, the learner conceives his TL surface 
coding in a topic-prominent manner. In a topic-prominent NL, a topic stands at the head of the 
discourse and the ensuing discourse is taken by the producer and the receiver to be related to it 
until a new topical referent is introduced. Subject-prominent languages like English, however, 
undergo a complex, involuted process of textual isation to achieve a tightly-spun, almost opaque 
surface structure. Discourse ties are usually representational and non-repetitive and so are very 
different in kind from the over-explicit and seemingly illogical (in English) use of certain transferred 
semantic markers commonly found in the writing of Chinese students. Consider in particular, 

and — i?^ (translated inaccurately as "moreover^ and "besides" respectively in 
many Chinese-English dictionaries) which stand at the head of a paragraph in Chinese and 
predetermine the topical range of the paragraph content. A tentative claim may be advanced that 
semantic markers are topical and explicit in Chinese whereas English, particularly in written mode, 
demands conformity to a code of implicit internal coherence and cohesion. 

Topic and Theme 

Halliday (op.cit.) has demonstrated convincingly that English information structure is theme, 
and not topic, driven. There are many different kinds of theme but all appear in sentence initial 
position. Among these thematic variants is topical theme; often a simple, one-part device using 
subject, complement or adjunct to convey ideational information: 

Topical Theme Rheme 

Marriage / cannot remain the same in Asia. 

More complex themes are the norm, however, in English once the frame of reference has 
been established. These typically deliver interpersonal and textual components to express viewpoint 
and form cohesive links within the discourse. 

Complex Theme Rheme 
I doubt very much whether marriage in Asia/can remain the same. 

Given-New 

In both examples above the Given-New contract is retained intact, but the topical themes 
found in Chinese, if transferred to English, always have the potential to suddenly introduce a new 
point of departure or frame of reference in a non-anaphoric way. 
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That said, whether the Given-New contract is broken or not depends on the position in the 
context at which a new topic is announced. Clearly if new referents introduce a focus of information 
not recoverable from the foregoing text, then the Given-New sequence will be reversed. The 
examples given so far from the experimental corpus do meet English Given-New expectations, 
because in each case the topic necessarily relates back to the title of the piece of writing. An 
example from the corpus that does break these expectations is this sentence which began a new 
paragraph immediately following on from a paragraph of discussion on the Asian woman's relative 
lack of freedom in the choosing of a husband: 

Topic/New 

For the Western women*s concept of freedom, 

Comment/Given 

it can allow them to make their own decisions who to marry. 

Rhetorically, expressing such a contrast often requires breaking Given-New u-xpectations to 
deliver the necessary sharp comparison with the preceding topic. However, Chinese writers in 
English tend to break the contract rather too frequently and probably unintentionally, conferring 
qualities of suddeness, overemphasis and perhaps incoherence to the text. 

Topics, including the extended, defined types already exemplified, are able to stand 
independent of, or absorb to a limited extent, other syntactic components. Complex themes in 
English, however, tend to aggregate other components together and assign overall primacy to the 
subject-verb arguments they subsume. Thematisation itself has cohesive, as well as expressive 
properties in that it provides a mode of binding ideational, interpersonal and textual elements tightly 
together. In so doing the information to be conveyed is made more implicit. Topic, in contrast, 
requires fewer components and ties in its composition since it has the function of making points of 
reference and ideational information focii immediately explicit. 

Interestingly the elicited corpus revealed evidence of reduced textual and interpersonal 
material. Expression of subjective viewpoint, for example, was remarkably infrequent and thus most 
texts conveyed an overall sense of depersonalisation. I am not putting fon/vard a Whorfian 
reductionist view that topicalisation and its associated simplification of thematic structure in English 
is the primary cause of this. It is perhaps more likely that this suppression of personal engagement 
in the world view is induced by historical patterns of cultural behaviour (Bond, 1986: 85) and that 
appropriate language devices evolved simply to convey rather than in any way determine, cultural 
thought patterns. 

Directions for Future Research 

It hardly needs to be pointed out how preliminary the investigations reported here are. The 
richness and complexity of this area of enquiry mean that any statements or findings must 
necessarily be treated as tentative and incomplete; the more so when the very small-scale nature of 
the empirical study carried out is recalled. Such a preliminary enquiry is bound to raise more 
questions than it can hope to satisfactorily address. Certainly most of the key issues remain 
unresolved. The following list might offer useful suggestions for directions of future research. Lines 
of enquiry might include: 

How the different properties of subject and topic might affect morphemic grammatical 
accuracy. For example, subject-verb agreement. 

Convergencies and divergencies of topic and theme, and differences in accent between 
topic-driven and theme-driven discourse. 
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How a newly-intrcxluced topic in context can break the given-new contract in English; in 
particular the effective range of this dislocation requires careful study. 

The use of pronominal anaphora by Chinese learners of English and its link to 
topic-comment organisation. In particular, the use of representational "it" rather than 
demonstrative "this " in the reference cohesion system of English. 

The connections between notions of markedness in written Chinese and English. A 
systematic analysis and comparison to detect similarities and differences might well shed a 
great deal of light on the interlanguage of Chinese learners of English. It might also be 
interesting, as a corollary to much of the discussion above, to seek evWence for the possible 
effects of English typological organisation on the Modern Standard Chinese writing of 
Cantonese users of English. 

The development of reference and instructional materials based on the principles of 
contrastive information structure and rhetoric to raise learner awareness of the problem. As 
LI and Thompson (1976) have made clear, existing reference materials have limitations: 

...it is often difficult to determine the typology of a language ... on the basis of reference 
grammars since such grammars are biased towards the subject-predicate analysis. 

Pedagogical approaches might profitably focus on contrastive consciousness-raising 
activities, initially at sentence level, since this is the starting point for the meaningful organisation of 
information. In the Hong Kong context this is important since many learners, even those at tertiary 
level, are unable to produce complex sentences with complete clausal embedding and 
intra-sentential cohesion. Work could then proceed to the higher levels of text macrostructure. This 
process would afford the ideal opportunity for fruitful co-operation between native and non-native 
teachers of English. 

This paper has attempted to adduce some initial evidence for the crosslinguistic influence of 
topic at intrasententiai, intersentential and paragraph levels In the written production of Cantonese 
learners and users of English. The importance of investigating the SLA tendencies of all dialectal 
groups of Chinese learners of English cannot be overstated. Chinese people constitute more than a 
quarter of the world's population, and the very strong Interest they show in learning English as a first 
foreign language means that key issues need to be resolved urgently to facilitate effective teaching 
and learning. Language typology and the effects of its transfer on written English may be one of 
these key issues. 
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NOTES 

1. By "discourse accent" I refer to characteristically non-native patterns of discourse organisation; for 
a more comprehensive account, see Scarcella (1983). 
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TO TEACH STANDARD REPORT WRITING 
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1. Introduction 

Until the 1980s, evaluation was a relatively neglected area in the field of language teaching. 
Mackay suggests a reason for the apparent lack of evaluation studies: 

•What may have contributed to the timidity regarding program evaluation were the great 
comparative studies carried out in the U.S.A. and U.K. to compare the effectiveness of 
different approaches to the teaching of ESL....The manifest lack of success of these 
studies in determining the relative effectiveness of the outcomes of different programs 
appears to have had the effect of convincing applied linguists that they should stay 
clear of such complicated and difficult-to-control activities." 

(Mackay, 1981: 91) 

However, in recent years, the importance of evaluation has been gradually acknowledged. 
Candlin, Kirkwood and Moore (1978) already include 'evaluation of students' and 'evaluation of 
course' in their model of course design. Stern (1984) includes evaluation as one of the 'curriculum 
processes'. Breen makes it the last level of his 'Process Syllabus', claiming that "on-goina evaluation 
of the teaching-learning process in the classroom would be a crucial activity..." (Breen, 1384: 58). 
Allen states that curriculum decision-making "must be subject to some form of evaluation, in order to 
check that the results of our decisions are meeting their stated objectives, and so that knowledge 
about curriculum processes can accumulate in a systematic and responsible way" (Allen, 1984: 70). 
Nunan considers evaluation to be "a necessary component in any curriculum plan" and "particularly 
important in a needs-based, learner-centred program which is directed towards the achievement of 
specific goals and objectives" (Nunan, 1984: 46). Brown argues that "evaluation should be the part 
of a curriculum that includes, connects and gives meaning to all of the other elements in a program" 
(Brown, 1989: 241). Elley stresses the important role of evaluation in education, stating that "if the 
quality of our educational programmes is to be enhanced, and we are to avoid following the 
misleading bandwagons of the past, it (evaluation) is an essential part of the enterprise of education" 
(Elley, 1989: 285). 

A lot of evaluation studies, especially the earlier ones, tend to be quantitative. This has 
aroused the concern of some practitioners in the field of language teaching. Lawton (1978) 
attributes the failure of past attempts to evaluate curriculum partly to the existence of an educational 
research climate that rewards accuracy of measurement and generality of theory. Paulston (1980) 
also argues that the quantitative paradigm, with its emphasis on objective, 'hard' and replicable data, 
is not sufficient by itself and needs to be supplemented by a more qualitative approach which would 
be process- rather than outcome-oriented. Other researchers would contend that quantitative and 
qualitative evaluation can co-exist. As Allen points out, it would be a mistake to think in terms of two 
clear-cut schools of curriculum evaluation: 
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"I am basically in sympathy with the belief that quantitative and qualitative research 
methods are not mutually exclusive, and that they throw useful light on one another 
when they are used in the same study." 

(Allen,1984: 73) 

The instruments used for evaluation need not therefore be confined to pre- and post-tests. 
Questionnaires, interviews and classroom observation can be employed to complement purely 
quantitative measures, though pre- and post-tests "should be an essential part of evaluation of all 
courses'* (McGinley, 1983: 91). The approach to curriculum evaluation thus moves from a purely 
product-oriented one to one that includes processes. Different stages of a curriculum are looked 
into. Such a shift from a product-oriented approach to a process-oriented one has been obvious in 
recent years. Various models have been developed to evaluate the different stages of a curriculum. 
Mackay (1981) proposes one which involves thirteen categories. This is a process-oriented 
approach which "avoids the pitfall into which evaluation studies frequently fall, namely, to focus 
exclusively on student product" (Mackay, 1981: 110). McGinley (1983) proposes a package which 
bears certain similarities to Mackay's paradigm but on a reduced scale. It involves the use of various 
kirxis of evaluative instruments. The Phi Delta Kappa National Study Committee on Evaluation 
develops the CIPP (acronym for Context-lnput-Process-Product) evaluation model which places 
equal emphasis on process and product. Breen (1989) also devises an evaluation cycle for the 
different phases of language learning tasks - task-as-workplan, task-in-process and task outcomes - 
and this model is evidently process-oriented. 

A model which enables us to evaluate the different levels of a programme is proposed by 
Johnson who points out that "Knowledge of products and in particular global products such as 
scores on proficiency tests, has limited value for decision-making unless evaluators know the 
learning processes by which the outcomes were achieved' (Johnson, 1989: 21). The four levels in 
Johnson's model are curriculum planning, ends/means specification, programme implementation 
and classroom implementation. The first level, curriculum planning, "consists of all those decisions 
taken before the development and implementation of the programme begins" (Johnson, 1989: 2). It 
is the policy-making level and determines the overall aims of the curriculum. The second level, 
ends/means specification, is Ihe process by which policy, and the means by which it is to be 
implemented, are operationally defined. Ends specifications should provide an exact characterisation 
of the target proficiency. Means specification should prescribe the method by which the target 
proficiency should be achieved" (Johnson, 1989: 4). This level is, in other words, concerned with 
syllabus writing - ends specification relates to objectives, and means specification to method. The 
third level, programme implementation, relates to "the development of teaching and learning 
resources, and the preparation of teachers to ensure that the resources are used effectively" 
(Johnson, 1989: 7). The fourth level, classroom implementation, relates to teaching and learning 
acts in the classroom. 

This paper will report on evaluation studies of two courses taught by the writer at the Hong 
Kong Polytechnic in 1989 (Fuller details of the methodology and results can be found in Leung 
(1990), unpublished M. Ed. dissertation). They were the Accountancy Year 1 (AC 1) and Company 
Secretaryship Year 1 (CS 1) English courses. The two sets of students followed the same English 
programme. Their ability level was comparable as both AC arxi CS courses were run by the 
Department of Accountancy and shared the same admission requirements. In fact, in 1990, the CS 
course was phased out and the Year 1 CS students, some of whom were subjects of this study, 
became Accountancy students when they were promoted to Year 2. Students on the course were 
taught how to write management reports of different formats - letter reports, memo reports, etc.; my 
research focused on standard report writing. The entire English programme was to be covered in 30 
hours, of which 14 hours were devoted to standard report writing. The duration of the whole 
programme was 10 weeks, and standard report writing was taught in the last 5 weeks. A seminar 
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mcxJe of teaching was adopted but I tried out an alternative problem-solving approach in one AC 
group and one CS group with a view to stimulating the thinking of the students. 

Before proceeding to the aim of my evaluation study, I would like to clarify the meanings of 
'standard report writing' and 'problem-solving approach' since the former rarely appears in the 
literature of report writing and the latter is open to many interpretations, but refers here to a 
methodology which was tried out at the level of classroom implementation. 



1.1 Standard Report Writing 

The familiar terms, long and short reports, seem to me to be a little confusing as "length is a 
relative term, and no specific page length can be agreed on as the breaking point between long and 
short reports" (Lewis and Baker, 1983: 203). Moreover, "short reports written in the traditional format 
tend to have more of the characteristics of long reports" (Lewis and Baker, 1983: 205). Some short 
reports "may run up to ten pages" (Brown, 1973: 42), or even "thirty pages" (Lewis and Baker, 1983: 
203). Thus, I decide to use the term 'standard report' instead to refer to the format of 'introduction - 
findings - conclusions - recommendations.' 



1.2 Problem-Solving Approach 

The term is often interpreted in different senses. In some literature, it takes on a relatively 
informal sense, while in others it is defined much more technically with explicitly stated strategies. 
"Some people define all thinking as problem-solving" (Brown, 1973). Fisher (1987) defines problem- 
solving as 'higher order' thinking, calling for skills such as inference, deduction, analysis and 
evaluation. According to Fisher, problem-solving involves both the critical and creative aspects of 
thinking. "The critical or analytical approach involves seeing the different parts of a problem and the 
ways in which they are related.. .the creative side is concerned with how to generate a variety of 
possible solutions, and the ways in which the problem might be tackled" (Fisher, 1987: 11). Treece 
(1985) lists various steps of problem solving including determining and defining the problem, 
collecting data, organizing data, evaluating and interpreting data and selecting a solution. 
Margerison (1974) gives a condensed version of these steps which he calls the PSA cycle: Problem 
diagnosis - solution development - action implementation. 

In two of my four groups, I tried out a problem-solving approach as a special methodology 
that gave prominence to some major activities and techniques. Brainstorming is one such technique 
that is often employed to increase the flow of ideas. The technique of questioning is important in the 
discussion stage (Fisher, 1987) and the skill of negotiating with others in groups is essential to the 
success of the problem-solving approach (Margerison, 1974). While implementing the problem- 
solving approach the teacher needs to be "gaining and keeping the pupils' attention, controlling the 
group, sticking to the point, and ensuring balanced participation" (Fisher, 1987: 37). It is also useful 
to get the students to report back to the class and share findings with other groups. 



2. Aims of Study 

The principal aim was to evaluate the effectiveness of the two English courses mentioned 
above. A further goal was to compare two forms of classroom implementation. A dual approach - 
quantitative and product-oriented studies together with qualitative and process-oriented methods ~ 
was employed. The students' pre- and post-teaching reports were compared. The coherence of 
each course was looked into by adopting Johnson's model of the four levels of evaluation described 
above. I hoped that my evaluation study would yield useful insights into the course design and 
teaching of standard report writing at tertiary level. 
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3. Methods 
3.1 Subjects 

Four groups of students helped me with my study - the AC 1 special methodology group 
(ACm), AC 1 normal teaching group (ACn), CS 1 special methodology group (CSm) and CS 1 
normal teaching group (CSn). ACm and ACn each consisted of 15 students. CSm consisted of 19 
and CSn 20. While special methodology refers to the problem-solving approach, normal teaching 
means the seminar mode. 



3.2 Procedure 

Information was gathered by means of pre- and post-tests, a questionnaire, interviews and 
classroom observation. 



3.2.1 Pre- and Post-Tests 

A pre-test was administered before the teaching of the standard report writing component, 
and a post-test at the end of the course. The tests were mainly concerned with the ends level 
(Johnson, 1989) and with the learning outcome of the students. In each test, the students were 
required to write a standard report within a time limit of 90 minutes. A test checklist based on the 
objectives of the course and to be completed by the teacher was devised to help determine the 
achievement of the objectives for standard report writing. 

A two-sample T-test was used to compare the mean improvement between ACm and ACn, 
and between CSm and CSn. The hypothesis was that the special methodology groups would make 
greater progress than the normal teaching groups. The mean improvement of ACn^ and CSm was 
also compared to find out if students of the two methodological groups nnade the same degree of 
progress. The expectation was that their degree of progress would be similar despite possible 
differences at their starting level. 



3.2.2 Questionnaire 

A questionnaire was administered at the end of the course to solicit the students' attitudes 
towards the standard report writing component of the course. The special methodology groups 
were requested to give their opinions of the problem-solving approach. 



3.2.3 Interviews 

Guided Interviews were conducted with the course coordinator of the parent department and 
with two students of the special methodology groups. Specific questions were prepared and asked 
in a fixed order. 



3.2.4 Classroom Observation 

Two lessons adopting the problem-solving approach (one ACm and the other CSm) were 
observed by a research assistant. Both lessons were conducted by the author and were on the 
same topic, using the same materials. An observation record form which provided an outline of what 
to observe was used by the observer. 
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4. Results 



4.1 Pre- and Post-Tests 

The post-test results showed that on the whole, the objectives of standard report writing 
were well achieved by all four groups. Scores on most areas were above 60%. Students achieved 
the best post-test results, with scores of over 80%, in the following areas: 

- Purpose of report 

- Selection of data 

- Grouping and sequencing of data 

- Precision with figures in describing data 

- Noting attitudes, expressing personal opinions and supporting opinions with data 

- Proposing immediate courses o'i action 

- Actions related to purpose and conclusions 

- Overall organization 



The following were the most problematic areas, with scores of below 30%, in the post-test: 

- When information was collected 

- Using sub-headings 

- Stating source and numbering of graphic aids 

- Highlighting salient features in graphic aids 

- Expressing different degrees of certainty 



Students made marked improvement (increments of approximately 40%) in the following areas: 

- Stating time in background information 

- Terms of reference 

- Use of instrument in gathering information 

- Use of headings 

- Selection, construction, captioning and labelling of graphic aids 

- Noting relationship 

- Summarizing findings 

- Numbering conclusions 

- Making recommendations 

- Overall organization 



Scores in both pre-test and post-test were poor, showing gains of only about 10%: 

- When information was collected 

- Use of sub-headings 

- Stating source of graphic aids 

- Highlighting salient features in aids 



The progress made by the special methodology group and the normal teaching group was 
compared to find out if differences at the classroom implementation level would affect the learning 
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outcome of the students. Therefore the pre- and post -test scores obtained by the students of ACm 
were compared with those of Group ACn, and scores obtained by students of CSm were compared 
with CSn. I also compared the progress made by ACm and CSm and tried to find out if the starting 
levels of both groups were the same, and if not, whether different starting levels would affect the 
effectiveness of the problem-solving approach. The table below presents results for all subjects. 



Table: Pre and Post-test results for all subjects in four groups 





ACm 


ACn 


CSm 


CSn 


Sub 


Pre . 


Post 


Imo 


Pre 


Post 


Imo 


Pre 


Post 


Imo 


_Pie _ 


Post 


Imo 


1 


25 


33 


8 


10 


25 


15 


17 


39 


22 


17 


33 


16 




11 


40 


29 


13 


35 


22 


10 


32 


22 


8 


36 


28 


3 


33 


43 


10 


14 


23 




14 


33 


19 


13 


40 


27 


4 


24 


36 


12 


15 


35 


20 


19 


30 


11 


14 


35 


21 


5 


19 


35 


16 


13 


26 


13 


22 


41 


19 


8 


34 


26 




20 


45 


25 


23 


37 


14 


15 


22 


7 


16 


29 


13 


7 


15 


37 


22 


21 


32 


11 


10 


32 


22 


11 


39 


2B 


8 


14 


40 


26 


11 


29 


18 


12 


23 


16 


17 


32 


15 




12 


42 


30 


13 


23 


10 


17 


37 


20 


30 


27 


-3 


10 


13 


37 


24 


19 


24 


5 


19 


40 


21 


17 


33 


16 


11 


10 


23 


18 


13 


28 


15 


8 


40 


32 


17 


40 


23 


12 


26 


40 


14 


11 


22 


11 


11 


18 


7 


15 


24 


9 


13 


16 


40 


22 


16 


34 


18 


11 


39 


28 


20 


30 


10 


14 


27 


37 


10 


16 


20 


4 


16 


47 


31 


13 


33 


20 


IS 


7 


30 


23 


16 


28 


12 


23 


45 


22 


14 


15 


1 


ie 


/ 




/ 


-J. 


u. 


/ 


16 


40 


24 


21 


32 


11 


17 


/ 


-J 


-J 


-J 


/ 


-J 


12 


46 


34 


14 


29 


15 


18 


y 


/ 


/ 


/ 


/ 


/ _ 


16 


49 


33 


15 


30 


15 


19 


i 


^ 


-J 





-U 





24 


35 


11 


11 


30 


19 


20 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


/ 


15 


29 


14 


mean 
imo 


19.27 


13.13 


21.11 


16.20 


5^^ 


52.21 


26.27 


67.88 


69.43 



(Key: Sub = subject; Pre = Pre-test; Post = Post-test; 
Imp = improvement) 



All four groups made progress on the pre/post-test measure. Results of the two sample T- 
test confirmed that the special methodology groups made better progress than the normal teaching 
groups, it can be seen that the mean improvements of the two special methodology groups, ACm 
and CSm, were similar. 
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4.2 Questionnaire 

In ACm, 13 questionnaires were returned out of 15 distributed; in ACn, 11 out of 15; in CSm, 
16 out of 19 and in CSn, 18 out of 20. Below are the results of the questionnaire. 



4.2.1 Course content 

Responses from all four groups concerning ^usefulness' were highly positive. Most areas 
were rated useful or very useful. The best-rated areas were organization of data and discussing and 
interpreting findings. The least favourable areas, on which students felt too much time had been 
spent, were related to graphic representations. 

The majority also wanted to see most areas maintained on the course. There was a 
substantial demand to extend the following: 

- Discussing and interpreting findings 

- Organisation of data 

- Presenting judgements 

The time spent on the standard report writing component of the course was rated just right 
by the majority of respondents. 

When asked what other aspects of standard report writing should be taught, some put down 
'layout* or 'format of presentation*. Some put down 'language* and 'jargon for report writing*. 



4.2.2 Course Materials 

The majority thought that just the right amount of materials was given on most areas but 
there were indications that further material in the following areas would be welcome: 

- Discussing and interpreting findings 

- Description of findings 

- Organization of findings 

When asked on what other aspects of standard report writing they wanted more handouts, 
many wrote 'None*. A few put down 'real examples of reports* and 'Standard report samples on 
different subject matters*. When asked how to improve the handouts, some put down 'Not enough 
practice work* or 'More exercises*, or 'More examples should be set*. 

Responses to usefulness of handouts were highly positive. The area which was rated least 
useful was highlighting salient features in graphic aids. 

Responses to the recommended reference books were least positive among all items in the 
questionnaire. Many left this section blank and a few even stated that they could not get hold of the 
books. 

4.2.3 Course Methodology 

Responses to the problem-solving approach from both ACm and CSm were favourable. 
Students found problem-solving strategies useful. More than 90% of ACm and almost 70% of CSm 
found that they were able to raise a lot of questions in their discussions. 85% of ACm and 75% of 
CSm found brainstorming a useful technique for generating kleas. Over 80% in both groups agreed 
that there was active exchange of ideas. All in ACm and 75% of CSm felt that they could generate 
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solutions. They had a sense of achievement when solutions were generated. 85% of ACm and 70% 
of CSm did not give up even when they encountered difficulties. 

Regarding the advantages of the approach, the majority of both ACm and CSm noted that 
they could learn through active participation. 85% of ACm and over 90% of CSm found that the 
approach made possible more student involvement. However, they were rather divided on whether 
participation was balanced in their discussions. About half in each group thought that participation 
was balanced but the other half thought that discussion was often dominated by one or two active 
members of the group. The use of a chairperson to regulate participation in group discussion was 
found to be effective by CSm (86% being positive about appointing a chairperson) but 30% of ACm 
had some reservations. 

Students agreed that they learnt through sharing with peers. All in ACm and over 90% of 
CSm thought they benefited from sharing. The students also found reporting back useful. 

About 80% of ACm and 90% of CSm found a lot of interaction among group members. 
Almost 90% of CSm and all in ACm found more teacher-student interaction here than the usual 
seminar mode. Consultation with teacher was considered useful by all in ACm and almost 80% of 
CSm. Students also found that they could learn in a more relaxed atmosphere under the problem- 
solving approach. Almost all in both ACm and CSm enjoyed the relaxed atmosphere of learning. 

The students also found the approach educationally useful. 85% of ACm and all in CSm 
thought that the approach could stimulate their thinking. In each group, over 80% thought that their 
faculty for critical thinking could be developed, and over 85% found the strategies helped their 
studies in general. 

In their responses to the open-ended question concerning the application of the approach to 
report writing, most were positive about its value. When asked which area/areas of standard report 
writing they considered to be best taught through the approach, answers on almost all areas were 
received. Some put down *AII'. Most put down interpretation of findings, conclusions and 
recommendations. When asked which area/areas should not be taught through the approach, most 
wrote 'None*. A few put down 'Introduction'. 



4.3 Interviews 

Both course co-ordinator and students perceived the need to include report writing in the 
English course. They also found the objectives highly relevant. 

For some objectives, the students had much previous knowledge. However, they found it 
helpful to consolidate their knowledge in this course. The teaching of use of graphic aids, in their 
opinions, should be the job of the parent department. They also thought that formal learning of 
report writing in the classroom was necessary. 

Regarding course structure, both co-ordinator and students thought that standard report 
writing was rightly placed at the end of the English course, after note-taking and summarizing skills, 
and after relatively simple report formats like letter and memo reports. They also thought that ten 
weeks was the right duration for the course, and that three lessons per week was not at all intensive. 

The handouts were found to be useful and sufficient on the whole. However, some areas 
were hardly touched on, e.g. highlighting salient features in graphic aids. Students were not totaHy 
satisfied with the design of some exercises which were too mechanical. They would like to see all 
five units developed on a central theme. 
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The students welcomed the problem-solving approach but they wanted to tackle real 
problems instead of doing artificial problem-solving exercises set in handouts. 

As far as learning is concerned, the course co-ordinator found that the students of both 
courses had improved their skills in standard report writing on the whole. Their strongest area was 
selection of data and overall organization. Progress relating to language accuracy was also obvious. 
However, he still thought that some were unable to draw sourxJ conclusions, arxJ they also needed 
to be more audience-oriented in their writing. 

The students thought that they learnt a great deal about the overall organization of a report, 
what to include in each section, how to write convincing conclusions, etc. Former Arts students 
benefited more in analysis of data than former Science students. The students would also like to 
write 'bigger reports* at the end of the course instead of one within a time limit of 90 minutes. 



4.4 Classroom Observation 

The time spent on various stages of the lesson was similar in both ACm and CSm lessons. 

In group discussion, students in CSm seemed to have more balanced participation. Many 
questions were raised and ideas generated. Members did not accept possible solutions readily; 
instead they tried to evaluate solutions and explore possibilities. The chairperson gave each member 
opportunities to express himself. The members were also able to apply certain strategies when they 
encountered problems - seeking help from the teacher, consulting reference books, etc. On the 
whole, they were actively involved in their discussions. 

In ACm, discussion tended to be dominated by two active members of the groups. The 
chairperson did not even give each member the opportunity to voice his opinions, nor did he 
attempt to sum up the ideas of the group. Viewpoints were sometimes given without elaboration. 
Nevertheless many questions were raised and Weas generated. 

Students of both ACm and CSm appeared to be quite independent in their discussion. They 
only sought help from the teacher occasionally. The teacher moved from group to group and 
offered help when she detected difficulties arising. 

Owing to time constraints, each group could not report back their discussion to the class 
thoroughly. Most groups just gave the answers to each exercise. Moreover, there was a great deal 
of overiapping when each group reported back on all the tasks they had completed. Generally 
speaking, the students seemed to find reporting back useful. 

Also because of limited time, the teacher could not give thorough feedback. She went over 
the answers of each exercise with the class and briefly commented on the differences of opinions. 
The teacher finally summed up the main points of the lesson. 

5. Discussion 

5.1 Pre- and Post-tests 

The fact that the vast majority of the objectives were well-achieved demonstrates that the 
course was generally effective. Of the objectives achieved, most were due to teaching although for 
some, such as noting attitudes, grouping and sequencing of data, the students had much previous 
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knowledge from secondary schooling. Of the four sections of a standard report, the objectives 
pertaining to recommendations were best achieved and the importance of this phase in a report 
appears to be widely recognized by students. Marked improvement was seen in the overall 
organization of their reports. In the pre-test, many of their reports contained just one section: 
discussion of findings. After the course, they learnt to present their ideas more systematically by 
dividing their reports into different sections and using appropriate headings for each section. 

Of the objectives that were not satisfactorily achieved, most were completely neglected by 
the students in the pre-test. However, their awareness of these objectives was not enhanced 
through teaching. Areas relating to graphic aids formed the majority of these poorly achieved 
objectives. The students were unable to select appropriate graphic aids to represent data visually; 
they were still rather ignorant of the rules for constructing these graphics, particularly pie charts. 
They still failed to give captions and sources of the aids and number them. In their discussion of 
these aids, they also failed to highlight salient features. Re-teaching of the poorly-achieved 
objectives was necessary, and re-examination of the materials used and time spent on such areas 
would be helpful. 

The result of the two-sample T-test also shows that differences at classroom implementation 
level affect the learning outcome of the students. Coherence of the first three levels, in my opinion, 
may not bring about the desired results if they are not echoed by the last level of the decision- 
making framework. 

The results also indicate that the problem-solving approach may be a more effective 
methodology for teaching standard report writing than the seminar mode. Problem-solving strategies 
like questioning, brainstorming and evaluating solutions are likely to stimulate more active thinking 
on the part of the students. 

The fact that there is no significant difference between the progress of ACm and CSm 
confirms my belief that the problem-solving approach works despite differences in the starting levels 
of the students. ACm students proved to have more previous knowledge of standard report writing 
than CSm students, as reflected by their pre-test scores. However, both these groups made more or 
less the same degree of progress on the course. 



5.2 Questionnaire 

The positive ratings on course content and methodology show that the bourse was generally 
well-received. Most of the course content was found to be useful. The least favourable areas were 
those relating to graphic aids, which received low ratings on usefulness and suggestions that they be 
cut back. 'Methods and source of information* was another area which received a significant 
demand for curtailment. The reason is, I think, different from that for graphic aids as reflected by 
ratings on usefulness. The students found this area useful but they wanted to have it cut down, 
probably because it was too simple and straightfonvard to be worth the amount of time spent on it. 

The result also shows that the present allocation of time to the standard report writing 
component (14 hours out of 30 hours) should be maintained. As for other areas which should be 
added to this component, the students* view of including more work specifically on points of 
language should be taken into consideration. In fact, language was taught throughout the course 
although it was not a specific objective of this component. Given more time, language could also be 
included in the syllabus. Besides language, they also put down 'layout* or 'format*. I interpret this as 
referring to the prefatory and appended parts and the physical presentation of the report. Again, 
these areas could be addressed if more time was given. 
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The students found the handouts generally adequate and sufficient. Perhaps more examples 
of reports ought to be included in the handouts as requested by some students. They also indicated 
that measures should be taken to ensure easy access to the recommended reference books. 

Responses from both ACm and CSm to the problem-solving approach were highly 
favourable but the results show that ACm students seemed to be able to exploit the problem-solving 
strategies better. The majority also understood the advantages of the approach. However, 
something should be done to ensure more balanced participation in group discussions. In fact, 
anticipating problems in this area, I appointed a chairperson to regulate participation. The different 
responses to this tactic from ACm and CSm, in my opinion, show that its effectiveness depends on 
the personality and capability of the chairperson. 



5.3 Interviews 

The existence of the course was well justified as both course co-ordinator and students 
recognized the need to teach accountancy students standard report writing. Accountants need to 
verify their inspection of a firm's financial records in the form of a report. They need to interpret and 
present information linguistically. The students also thought that mastery of the report writing skills 
was more effective through formal learning than through reading references. The course was also 
coherent at ends-means specification level as the objectives were considered to be highly relevant 
and the course structure appropriate by both course co-ordinator and student. The students' 
opinion that the parent department should teach the use of graphic aids is, I think, justifiable since it 
is not primarily a question of language. The present course structure of three lessons a week 
spreading over ten weeks helped to establish better teacher-student rapport than a course, 
spreading over thirty weeks, with only one lesson per week. 

The interviews reflect that the handouts ought to be slightly revised as some exercises like 
sentence transformations were too mechanical. A thematic approach to the five units of the 
handouts (overall organization, introduction, findings, conclusions and recommendations) would be 
desirable. 

The interviews also reveal a shortcoming of the problem-solving approach implemented in 
this programme, as students were simply asked to deal with artificial problem-solving situations set in 
the handouts. 

The students' preference for writing 'bigger reports' instead of one completed within ninety 
minutes should be given serious thought. 'Bigger reports' are likely to be more challenging to the 
students and perceived as more authentic. 



5.4 Classroom Observation 

The teacher's explanation of the lesson objectives helped to make the students' learning 
more purposeful, as the students could be more aware of what they were expected to achieve in the 
lesson. Clear instructions for the completion of the various tasks were also useful. 

The students obviously benefited a great deal from the problem-solving approach. Both the 
critical and creative aspects of their thinking could be stimulated. They were able to pool together 
ideas and critique them, generate solutions to problems, evaluate solutions and explore alternatives. 
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6. Conclusions 

The standard report writing component of the course was found to be effective and coherent 
on the whole. 



6.1 Course Effectiveness 

The vast majority of the objectives were well achieved with the exception of those relating to 
graphic aids. Re-teaching of these poorly-achieved objectives was definitely necessary. 

The effectiveness of the course was also reflected by the students' progress. A comparison 
between the pre and post-test shows that the students had benefited from the course. The two- 
sample T-test also shows that the special methodology groups had made better progress than the 
normal teaching groups, while the two special methodology groups did not show much difference 
between their progress. Hence, I can conclude that differences at classroom implementation level 
do affect the learning outcomes of the students, and that the problem-solving approach may be an 
appropriate methodology to teach standard report writing. The problem-solving approach works 
equally welt for students with different starting levels. 



6.2 Coherence 

The standard report writing component of the AC and OS course demonstrated a high 
degree of coherence, despite a slight mismatch at certain points of the decision-making framework 
(Johnson. 1989). 

The policy decision at curriculum planning level to include standard report writing in this 
course is correct, as it caters to the needs of society. The social needs for report writing are 
obvious, and increasing in today's business worid as an effective means of communication. The 
inclusion of report writing in this course also caters to the needs of the students who followed the 
old Use of English syllabus in the Matriculation course. They had very little, if any, previous 
knowledge in report writing. 

The objectives of the standard report writing component were also clearly spelt out and 
found to be useful. They were well achieved by the students of both AC and CS courses. The 
course content was received favourably. The course structure was also found to be an appropriate 
means to achieve the ends. So there was coherence within the ends/means specification level. 

Coherence between the curriculum planning level and ends/means specifications level was 
to a large extent achieved as the social needs were spelt out cleariy at the ends level, and the 
course structure was able to achieve the ends and meet the needs specified at the planning level. 
However, the needs of society and students would be best answered if the syllabus could be 
rewritten to include real problems and the course structure modified so as to allow the students 
more time to tackle these problems. 

Regarding programme implementation, the materials used on the course were found to be 
largely relevant to the objectives. Only a few relating to graphic aids were neglected. However, 
some exercises were not intellectually challenging enough for tertiary students. Hence, there was 
some mismatch between this level and the curriculum planning level as policy makers intend the 
course to produce students who could think critically. 



76 



HONGKONG PAP£RS IN UNGUtSTiCS AND IANQUAG£ TBACHiNG U (1991) 



At classroom implementation level, the problem-solving approach was found to be an 
appropriate approach for teaching report writing. It was also consistent with course specifications at 
the curriculum planning level as it catered to the society's demand for people who could think and 
tackle problems. The students could apply the problem-solving strategies to daily-life situations. 
Coherence was also achieved between ends/means specification and classroom implementation to a 
large extent. The explanation of the lesson objectives at the beginning of the lesson served to 
establish a link between the two levels. The content taught in each lesson was closely related to the 
course objectives. 

Only slight mismatch was detected between classroom implementation and programme 
implementation level. The teachers were not guided on how to use the materials. Lesson plans 
were not given to promote uniformity at classroom implementation level. 



7. Recommendations 

7.1 Recommendations for Standard Report Writing Programmes 

To design an effective standard report writing programme, I think the first consideration is to 
respond to the needs of society and of students. As society is constantly undergoing changes, its 
needs are also changing all the tin?e. Besides, the students' needs are constantly changing too. 
Students entering Hong Kong tertiary institutions before the academic year 1988/89 may have had a 
greater need to learn how to write reports. However, with the implementation in 1989 of a new Use 
of English syllabus which includes practical skills like memo and report writing, students may have 
less need to learn report writing. Thus I suggest that the needs of society and students should be 
constantly reviewed. 

To make report writing more purposeful, I recommend that an industrial placement be 
incorporated into a standard report writing programme during which they can have an opportunity to 
identify real problems in their working environment, carry out primary research and collect relevant 
data. Such an arrangement would allow for more thorough application of the problem-solving 
strategies and be likely to offer more challenges to the students. 

To ensure that the materials contribute to the achievement of the objectives, the syllabus 
writer should help to develop the materials. He need not be the only person responsible for 
materials development. In fact, a team can be formed to develop, review and revise the materials. 
To help the teachers exploit the materials effectively, tutors' notes would be helpful. 

At classroom implementation level, I would recommend approaches which are likely to 
stimulate active thinking, generate more student involvement, encourage sharing of Weas 
and more interaction with peers and teacher. The problem-solving approach is one such approach, 
and I think it is particularly useful for teaching such areas in report writing as interpretation of data, 
drawing conclusions and making recommendations. 

7.2 Recommendations for Other Evaluation Studies 

For future evaluation studies, I would recommend that a detailed checklist be devised for 
classroom observation. In fact, observation checklists are often used in evaluation studies but in my 
own study I deliberately experimented with a classroom observation form which outlined the different 
stages of a lesson. I expected such an observation record form to be able to provide the observer 
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with more flexibility in writing their comments. However, this may only be useful to an experienced 
observer; it is not likely to be of great help to an inexperienced one such as the observer of my 
lessons, who is a research assistant. Even with an experienced observer, prejudice may come in 
when such a brief observation form is used. Thus, to ensure objectivity, I would prefer the use of a 
detailed checklist. 

I have also found it useful to combine the product and process-oriented approach in course 
evaluation. We should not only be concerned with the final outcome of the course but also with 
every level of decision-making. Adopting a process-oriented approach, we can kientify mismatches 
within each level and between the various levels, and then attempt to rectify them. 

Finally, I would like to reiterate the importance of incorporating evaluation in any curriculum 
design. I am convinced that ail courses should be evaluated, as fully as time and human resources 
permit, and I also hope more evaluation models will be proposed which make possible more valid 
evaluations. 
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Introduction 

In November 1990, the Hong Kong Government's Education Department released the 4th in 
its series of Education Reports (ECR4) prepared by specially-appointed Education Commissions. 
The ECR4, on Language Improvement Measures for Schools, is already notorious locally for its 
proposal that government and government-subsidised secondary schools adopt a language 
streaming policy, using results of an evaluative instrument at Primary 6 to decide which children 
would benefit from an English-medium secondary education, and which should continue with 
Chinese-medium education. This paper explores the implications of this recommendation from a 
tertiary perspective, and questions the extent to which the Commission has properly addressed the 
problem of equality of opportunity for children issuing from the Chinese-medium stream after 1994. 
The paper goes on to attribute the evolution of this policy to a widespread lack of understanding of 
the nature of second-language-medium academic education. This paper is short and at times 
polemical, but its main arguments have been carefully considered; fuller discussion of background 
issues can be found in Bruce (1990) and Lewkowicz (1990)\ 



The Problem: Educational Outcome or Process? 

One of the driving motives behind the work of the Education Commission has been the 
desire to create a more equitable distribution of educational and vocational opportunities among 
Hong Kong's secondary school population ~ a utilitarian desire for quality and equality. More 
specifically, the Commission has undertaken the unenviable task of attempting to enhance the quality 
of the 'means* or process of education for the majority of children, while preserving the quality and 
equality of the outcome or 'ends' of that process. Regrettably, in attempting to prescribe remedies 
for the primary and secondary school sectors, the Commission seemed constrained to 'respect the 
autonomy' of the tertiary institutions, to the extent that they have been forced to pursue justice under 
the kind of constraint that utterly confounded Shylock. In an educational system whose tertiary 
sector is predominantly English medium, linguistic and academic competence are as inextricable as 
flesh and blood; secondary and tertiary education cannot be segregated when making decisions 
relating to the development of those competences. It is not enough for the Commission to state 
(p. 11 7) 'We understand that the Government intends to discuss with the tertiary institutions' those 
means by which equality of access to tertiary education can be assured. 

Policies aimed at tackling shortcomings of both quality and equality will always risk 
producing the kind of volatile cocktail the Commission has come up with, especially tf equality is 
made to trail behind prescriptions for improved quality. The consumer in Hong Kong is concerned 
with opportunity, and the freedom to pursue that opportunity for his or her child. The Education 
Department should have been able to predict the futility of a prescription which is directed at 
improving both quality and equality in secondary education but which is powerless to ensure equality 
of value of the product, in terms of access to the most profitable and prestigious career 
opportunities. In declaring that the majority of students 'would benefit more from their education if 
they were to learn in their mother tongue' (p.102), the Commission is unlikely to be referring to 
educational 'benefit' in terms that most parents or major employers would understand. The parent 
population and employment sectors in Hong Kong are not unique in tending to be concerned 
primarily with 'optimal proven outcomes' (qualifications testifying to superior ability) rather than the 
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intrinsic quality of an education. In Hong Kong, there may well be a basic respect for any student*s 
ability to persevere and triumph at the highest level in what is a foreign-language-medium system, 
but this is a respect not untinged by cynicism. Centralised curricula and examination systems all too 
easily develop into highly prescriptive and conservative vehicles for the promotion of conformity, 
acquiescence and emulation. Paradoxically, in seeking to guarantee fairness and reliability in their 
reward system, national examination syndicates are drawn into setting the kinds of examination 
which inevitably privilege norm over criterion referencing, with closed-ended questions breeding in 
students a belief in the rightness and wrongness of things, and in teachers a disposition to 
prescription. This not only militates against an intrinsically worthwhile education, but sorves to 
encourage a hierarchical view of education and, in Hong Kong, largely determines access to and 
exclusion from an English-medium education. Since all tertiary institutions continue to be adamant 
in requiring certifiable proof of both academic and English competence from all candkJates, parents 
need little assistance in making a 'rational and informed choice as to which medium would suit their 
children best' (p. 103). It is clear that equality at the end of the secondary 'tunnel' is what is 
perceived as paramount; despite the Commission's recognition of the 'need to ensure that students 
learning through Chinese ... are not disadvantaged' (p.116), an assurance of this equality is what 
continues to be glaringly absent in the recommendations of the Report. 



Basis for a Solution 

The easiest solution would be to continue the current free-for-all, which at least has the merit 
of leaving discrimination to market forces, that is, to the parents' ability to get their children into the 
right schools, and to the prevailing open access to academic achievement through the examination 
system. The majority fail to glean much profit from the experience, and are branded as relative 
failures, but which 'advanced' western education system is free from the same accusation? While 
the status quo may be a continuation of a colonial system, it does have the merit of being relatively 
'democratically-driven'. Given the way the Report's recommendations continue to privilege 
English-medium schooling, while narrowing the majority of parents' and children's options from an 
early stage, it is easy to understand their largely hostile reception. My proposals below emanate 
from the belief that it is possible to achieve greater quality and equality both of the process of 
education and of career opportunities arising out of that education process. 

Any such proposal requires a fundamental change of perspective. Since parents and the 
employment sector are concerned with outcomes, education needs to be looked at more honestly 
as being 'end-driven*. That is, the higher and further educational and vocational or professional 
opportunities - on which most parents have set their sights long before their children reach 
secondary school - must become the starting point in any analysis and review of the school 
curriculum. If equality is to be properly recognised as determinant of quality, then the system should 
be geared to providing equality of opportunity at the exit point of secondary education, rather than 
at the entry or any intermediate point. I shall argue that the heat being generated in debate over the 
appropriate duration of an English for Academic purposes brkJging course is energy subjected to 
futile combustion. 



Token Bridging or the Real Thing: Hobson's Choice 

The Education Department (ED) has recommended provkiing an English bridging 
programme 'between May and the end of August' (p.1 17) at the end of Form 7 for all those 
Chinese-medium students who have been offered provisional acceptance by a tertiary institution. As 
a move to smooth the path of a linguistically-precocious elite, this could prove an effective as well as 
economical policy However, if, as the ECR4 would pretend, the intention is to provkJe equal 
opportunity to the 70% of the secondary school intake likely to comprise the Chinese-medium 
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Stream, the Commission has clearly been ignoring the warnings of local English language educators 
and experts. Most tertiary language learning specialists have recommended a full academic year for 
the provision of an adequate bridging course for students whose only exposure to English would be 
in studying the language as another separate subject^. This advice has been offered faute de mieux, 
on the assumption that only the form of the post-' A' level bridging programme was negotiable; it did 
not issue from discussion as to whether such a programme was desirable in the first place. If 
provided as a 'posthumous' supplement to their curriculum, a whole-year bridging programme would 
put the Chinese-medium students a full year behind their English-medium counterparts. By 
dismissing this option out of hand, the Government has probably ensured that none of the bridging 
options will be taken up, and that a broader re-examination of policy options will have to take place. 
Would that their rationale were related In some way to the question of equality - an objection, for 
instance, to arbitrarily shortening one group of children's earning life by a year; sadly, the rejection 
seems motivated rather by questions of finance and politics: the expense of an additional year for 
70% of the secondary school population, and the fact that the proposal smacks of a tertiary 
Foundation Year being reintroduced through the back door. Its position once stated, the ED is 
clearly not for turning. 

Out of this possible impasse may spring some hope for a more equitable bilingual education 
policy. It is clear to all concerned that equality of opportunity in the face of an English-medium 
tertiary sector and English-rewarding employment sector cannot be bought easily or cheaply. The 
optional solutions offered here require recognition of a principle that the Commission has already 
accepted in its rationale for Target-related assessment: that levels of academic (and linguistic) 
attainment are not rigidly tied to age or age groups. In both alternative 'bridging' proposals offered 
below, I advocate a system which is geared to ensuring that children exit from both English-medium 
and Chinese-medium schools with the same level of linguistic and academic preparedness, and 
therefore opportunity, and at the same age. This last point is crucial; educationists seem too ready 
to talk of additional years for bridging courses. Parents will perceive only the building of a further 
dimension of inequality into the educational system. This system at least currently offers their 
children open, albeit 'handicapped'^, competition with each other right up to the final 'A' level hurdle. 



Two Proposals 

1. A *Head-Start' Scheme 

The ECR4 proposes assigning schoolchildren to English and Chinese-medium streams at the 
end of Primary 6, as part of the exercise of allocation to a secondary school. A scheme of 
target-related assessments is proposed, in English, Chinese and nnathematics, to determine which 
children gain entry to an English-medium stream. The stigma of failure at this juncture cannot easily 
be attenuated by talk of what is educationally 'best' for a child. This alternative proposal advocates 
the delay of decisions on streaming to the end of secondary Form 2. Not only would this avert 
enormous psychological pressure being applied to children at a transitional point in their education, 
but it would fall at the juncture in the secondary curriculum when pupils, and their parents, are 
deciding on their '0' level programme and orientation. As a result of target-related assessment at 
the end of Form 2, pupils allocated places in the English-medium stream should be put through a 
bridging year aimed at preparing them for a 2-year 'O' level curriculum in the exclusive medium of 
English. Pupils allocated to the Chinese-medium stream would commence immediately with their 'O' 
level studies in Secondary 3, following a path which would take them to the equivalent of 'O' level 
and 'A' or 'AS' level"* one year ahead of the current schedule, giving them a 'head start' on the 
English-medium stream students. 
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By this system, Chinese-medium 'O' level graduates would have the option of continuing to 
'A' level or leaving school to take up further education or full employment opportunities, while 'A* 
level graduates (one year ahead of their English-medium counterparts) would have the option of an 
academically and/or vocationally-oriented English-rich bridging programme, followed by entry into 
either tertiary education or the job marketplace. Since the Government has calculated that, by 
1994-95, there will be first degree places in tertiary education for four out of five 'A' level 
matriculants, and given the proportion who will continue to seek tertiary education outside the 
territory, we can anticipate that the overwhelming demand would be for the academic route. The 
Chinese-medium 'O' level syllabus would ideally featuie a) a strong English language component, 
and b) one examinable but non-critical subject in the medium of English, to provide students with 
minimal but essential exposure to English as a learning medium. As I suggest later, this could well 
be the ill-fated Liberal Studies 'AS' course. The post-'A' level 'bridging' programme would continue 
the development of students' conceptual and rhetorical skills, but in the medium of English. Any 
gaps in subject knowledge base resulting from the compression of the curriculum, and its gearing to 
the average student capability, could be made up in this bridging year. By this system, the offer of 
tertiary places would be based on 'A' and 'AS' level results. The offer to Chinese-stream matricuhnts 
would be conditional on their satisfying an English requirement at the end of the 'bridging' year - 
currently grade *D' on the Use of English exam. 

The principle premise behind this proposal is that even the majority of the top 30% of the 
current crop of English-medium school matriculants entering university - those destined for the 
estimated 30% place allocation to English-medium schooling in 1994 - lack the necessary English 
and study skills to cope with an English-medium tertiary education in Hong Kong^ The proposed 
antidote is to use the year before the 'O' level syllabus commences (3rd year) to concenirate on the 
medium of study at the expense of new examinable content. This approach is based on the belief 
that not only can the system accommodate different rates of learning, but that the substantive 
(subject-based) agenda can vary across, for example, language streams, as long as it serves more 
transferrable and durable educational objectives, such as the development of analytical and critical 
thinking, and powers of argumentation and speculation. 

A valid criticism of this proposal is that it would seem to ignore the high correlation between 
linguistic proficiency - specifically English language proficiency - and academic attainment. 
Children gaining entry to the English-medium stream are likely to be academically precocious; 
having them study the same subject matter one year behind their Chinese-medium counterparts 
might be interpreted as gross underexploitation of their learning potential. The alternative proposal 
offered below is a concession to this objection, but it is, in my opinion, the weaker option, 
constituting a concession to a traditional abhorrence of any retardation to the education of the 
'gifted' child. 



2. An Alternative 6th-Form Agenda 

Students in both streams would follow the same curriculum, in parallel, as far as 'O' level, 
after which Chinese-medium stream students would only take 'AS* or 'H' levels, examinable at the 
end of Form 6. Tertiary places would be offered on the strength of those results - provisional on the 
satisfactory completion of the same 'bridging' year as proposed above. There would still be a 
uniform entry point for tertiary education - post-Form 7 - with a bridging year for Chinese-stream 
students, offering preparation for either tertiary education or the business and technological 
professions. 

While this option would require a more systematic effort to bring Chinese-medium students' 
knowledge base up to that of their English-medium counterparts, it does have the merit of operating 
with an established curriculum, that used by Chinese-medium schools preparing students for the 
Chinese University of Hong Kong. This close correspondence to the system currently practised by 
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the Chinese University makes this alternative more practicable. It might, then, be considered 
premature that the one-year 'H' level is to be phased out in 1992 (H.K. Education Department, 1989), 
given that these qualifications are currently deemed satisfactory for entry to Scottish Universities 
(which also operate a 'Higher' certificate exam system), subject to proof of competence in English. 
This proposal appeals less to the principle of different learning rates or stages, but like the first 
proposal does subordinate curricular 'content' to what we have identified as more transferrable and 
durable educational objectives, the development of those intellectual and communicative powers 
without which content knowledge remains inert and undigested by the learner. 



Implications for the Tertiary Sector 

The University of Hong Kong has already informed the Government of its own diagnosis of 
the language in education problem, in its rationale for a Foundation Year (University of Hong Kong, 
1988). Neither the English or Chinese-medium systems has been able to prepare students 
adequately for academic study. Now the Government has decided that the number of first degree 
places must be increased by 100% by 1997. The fact that it has also suggested that this can be 
done 'without sacrificing quality' (p.3) and, as recent indications from the UPGC make clear, by 
increasing the staff-student ratio, indicates a worrying lack of awareness of the nature and scale of 
the language problem in tertiary education (see Wong, 1984; So, 1988). The Commission's 
prescriptions for English include exhortations to the tertiary sector to 'do more to strengthen the 
current English improvement courses' (p. 117) for undergraduates; yet all the signs point to 
government funding of English training at tertiary level being cut back to a fraction of what was being 
promised a year ago. 

The real danger in the Government's effectively minimising the responsibility of the tertiary 
institutions for tackling the English problem is that the tertiary sector will be encouraged to continue 
to live with an ill-defined second-language-medium identity. Where the secondary sector takes its 
references largely from within Hong Kong, its educational targets being effectively set by the 
Territory's own tertiary institutions, the tertiary sector is obliged to measure itself against the 
advanced first-language-medium tertiary systems of the United Kingdom, the United States and the 
Commonwealth. It does not require advanced arithmetic to discern the improbability of attempting, 
in a second language, to emulate the standards of those institutions within a 3-year degree 
programme, at a time when increasing numbers of 'first language* universities in the U.K. are opting 
for 4-year undergraduate programmes. As long as the curricula remain unashamedly content-based, 
and funds to develop English training - the medium through which this content is mediated - 
continue to be cut back, questions have to be asked about the q'.^-^lity of tertiary education being 
offered in Hong Kong. The tertiary institutions are equally guilty of aeing seduced by the veneer 
rather than the substance of students' intellectual endeavour. Recent attempts to introduce a Liberal 
Studies AS level subject into the 6th form curriculum, and have it accepted as a compulsory subject 
for university entry, have met with an unsympathetic reception from both University of Hong Kong 
and Hong Kong University of Science and Technology; neither is prepared to make it compulsory, 
and this effectively will discourage schools from offering the course, and students from taking it. 
(For a more optimistic prognosis, compare Falvey, this volume]. There remains some hope that, 
since the Liberal Studies syllabus aims to develop academic and intellectual skills, rather than teach 
a body of knowledge, its future might well lie in a tertiary bridging programme for Chinese-medium 
students after 1994. 



Conclusion 

There is an overwhelming case for an Education Commission to be established to conduct a 
wide-ranging examination of the provision of tertiary English-medium education in Hong Kong. 
Some of its likely brief should have fallen within the compass of this Commission, which should, for 
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example, have distinguished between the learning needs and conditions characteristic of first-and 
second-language-medium education systems ~ at both secondary and tertiary levels. The 
Commission's attempt to exclude consideration of the tertiary sector from practically all areas of its 
deliberations has resulted in one of the major weaknesses of the Report. 

The evidence points to Chinese-medium matriculants becoming the victims of a 
parsimonious and almost casual Government policy towards the language problems they will be 
facing in tertiary education later in the 1990's. The use of the term 'remedial' as a hardy perennial to 
dismiss the claims of English to serious funding at tertiary level is indicative of the lack of 
understanding of the role of language in second-language-medium academic education. Unless 
there is radical re-thinking of this issue by the Government, the kinds of reforms being entertained in 
the Report - and in my suggestions above - will not make a significant contribution to the problem 
of student aptitude for English-medium tertiary education in Hong Kong. 



NOTES 



1. Bruce (1990) deals with some of the problems of running an L2-medium education system, with 
special reference to Hong Kong. Lewkowicz (1990) summarises the broader criticisms of the 'eCR4 
report made by staff of the Language Centre of University of Hong Kong, and records the responses 
to those criticisms offered by a leading member of the Education Commission. 

2. See Lewkowicz (1990), summarising HKU Language Centre's response to the 1989 'white paper' 
version of the Education Commission's 4th Report. Faculty of Education, University of Hong Kong, 
in its response to the final ECR4 (Crawford and Hui 1991) also describes the proposed three-month 
programme as 'totally inadequate' and, at a special colloquium held to debate options for a 
secondary-to-tertiary bridging programme (20/12/90 at the HK Convention Centre), secondary and 
tertiary English teachers voted almost unanimously in favour of a minimum one-year brkJging course. 

3. A major handicapping factor is the ability of the different schools to provWe effective 
English-medium instruction, and the Commission has made a number of recommendations to 
remedy this problem. 

4. Latest figures on Use of English grades of University of Hong Kong Ist-year entrants show that 
only 1 7.6% score A or B on that examination. 

5. 'AS' levels are 'supplementary advanced' level courses, intended to be taken in a single academic 
year, much like the 'H' level which is soon to be phased out, and which corresponds closely to the 
Scottish 'Higher' level 6th-form course. 
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Introduction 

Hong Kong is currently undergoing a series of examination and curriculum innovations which 
could have far-reaching effects on the practice of teaching in upper secondary forms, in 1994 the 
new series of Advanced Supplementary (A/S) examinations will be introduced. These examinations, 
designed to provide a broader sixth form curriculum (Grade 12) than the existing narrow one, will be 
available in a wide range of subjects. 

Two major innovations will be the introduction of Liberal Studies and Chinese Language and 
Culture. Liberal Studies has been introduced in order to broaden the outlook of Hong Kong 
students. The introduction of Chinese Language ^nd Culture is Intended to enable students to 
appreciate their cultural and linguistic heritage. It is eventually hoped that Liberal Studies, Chinese 
Language and Culture and the Use of English examinations will become compulsory subjects for all 
prospective entrants to tertiary education. The Use of English examination is already compulsory for 
University entrance. 

Some view it as unfortunate that a successful rearguard action has been fought by the 
traditional 'hard' disciplines in the tertiary institutions - those with already overcrowded curricula 
(Medicine and Engineering). They have managed to convince the authorities to make Liberal Studies 
and Chinese Language and Culture optional, not compulsory, subjects. Undoubtedly, these attitudes 
will soften in the next few years. 

In this paper the syllabus for the new A/S English Literature option will be considered. The 
curriculum innovations in this option and the changes in teacher practice which will result from those 
innovations may have beneficial results for the tertiary institutions which admit the students who take 
the option. 



Composition Theory and Practice 

The last decade has seen much discussion and a growing amount of research into the issue 
of how writers compose ext. Early work by Emig (1971). responding to a call by Braddock, 
Uoyd-Jones and Schoer (1 963) for research into LI (First Language) writing processes and 
subsequent work by Perl (1978), Pianko (1979) and Faigley and Witte (1981, 1984) into the 
composing processes of first language writers revealed that writing is not linear, that it is recursive, 
and that substantial rewriting and revision takes place. Such pioneering work with first language 
writers was followed by research into the composing processes of second language writers by 
researchers such as Chelala (1981), Zamel (1982, 1983), Jones (1982, 1985), Raimes (1985), Brooks 
(1985), Rorchach (1986), and Martin-Betancourt (1986). Studies involving Chinese writers have been 
conducted by Lay (1982), Arndt (1987) and Friedlander (1990). 

Some of the major findings of this research are that: 

- lack of competence comes more from lack of composing 
competence than from lack of linguistic competence 
(Jones 1982, Zamel 1982, Raimes 1985) 
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- the composing processes of 'unskilled' L2 writers and 
'unskilled' L1 writers are similar (Zamel 1983). 

- competent L1 writers use cjmmon strategies. 

Out of this research came changes in the theory and practice of composition teaching to 
first language writers and, subsequently, to second language writers. In first language writing one 
can trace the development of the 'process approach' over the past decade (see table below and 
subsequent text developed from Johns 1990: 25-31). 



Table: Process Approaches 



EXPRESSIVISTS 




COGNITIVISTS 


WRITER AS 


WRITER AS 


SOCIAL 


CREATOR 


INTERACTANT 


CONSTRUCTIONIST 






VIEW 



The 'Expressivists' were characterised by an approach which emphasised the power of the 
student as writer providing opportunities for the individual and uninhibited expression of honest and 
personal thought. Journal writing was a typical technique used in this approach which still has 
powerful adherents (Elbow 1973, 1981a, 1981b). 

The 'Cognitivists* are characterised by an approach which treats writing as a problem solving 
activity. This approach has had a great effect on L2 research, theory and practice. It emphasises 
the importance of the concepts of THINKING and PROCESS. 

'Writer as Creator' - Flower (1989) and Hayes and Flower (1983) look at the writer as a 
creator - one who plans extensively and 'defines the rhetorical problem*. Techniques used include 
group work in planning and writing, the creation of several drafts, and paper revision at the macro 
level (not merely 'polishing' text at the syntactic or morphological level but altering it in the areas of 
ideation, rhetoric and coherence). 

Writer as Interactant - Bakhtin (1973) stressed the need for the writer to interact with the 
audience. The notion that writing in English should be 'writer responsible' has now become part of 
current L2 writing theory and practice. 'Writer responsibility' can be defined as the acceptance by 
the writer that it is his/her responsibility for communicating with the reader; providing clear 
arguments, demonstrating opinions and their organization; and revealing the form of the text and the 
content in a manner accessible to the reader. 

Social Constructionist View - This view considers writing as a social act that can take place 
only within and for a specific context and audience. This approach stresses the importance of using 
the appropriate genre, knowing who the audience is, and writing for and within a 'discourse 
community'. This is the most recent trend within the cognitivist view of process in writing and is 
documented in the work on genre of Swales (1990), Dudley-Evans (1987), Martin (1985) and 
Berry(1989). 
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One can see within these developments the rationale for the theories and techniques in 
teaching L2 composition which have emerged in recent years: 

- clear identification of the purpose for the communication 

- writer responsibility towards the audience 

- awareness of the genre to be used 

- a well-signalled rhetorical framework 

- an identified audience 

- an acceptance of the notion of drafting and revising 

It is believed, by advocates of the process approach that these, together with an acceptance 
of the role of the reader as a genuine source of feedback, help to provide the supportive framework 
within which students can learn to develop ideas coherently. This approach also reflects the world 
of academic and business writing where written drafts are created and discussed and rewritten 
regularly. (Given the above it is sad to see that the questions set in the most recent Composition 
Paper in the Hong Kong Use of English Examination (Summer 1991) eschew these trends and return 
to an audienceless, non-formatted, purposeless and artificial batch of questions which has caused 
concern among many of those involved in the teaching of writing, leading to a number of letters to 
the press.) 



The A/S English Literature Syllabus - Description 

The new English Literature A/S curriculum contains many features which distinguish it from 
traditional syllabuses in this subject. It conforms to the sixth form curriculum guidelines established 
by the Sixth Form Working Party on Curriculum Development in trying to expose students to a range 
of ideas in order to broaden their education. Three major curriculum developmental aims were 
adhered to: 

- A full range of student needs should be taken into account - not merely intellectual 
ones. Social and communicative skills have to be developed so that students may 
become well-informed people who can enjoy and apply themselves to their future 
careers with confidence and ability. 

- The curriculum and syllabus should be appealing and accessible to both Arts and 
Science students. 

- The syllabus should discourage passive, memorised work and should encourage 
students to become actively involved in the learning process. 

The syllabus does not attempt to provide a narrow focus for detailed literary study of a very 
few books. Instead it addresses the complementary media of print and film/video in order to expose 
students to a number of themes so that they can develop their own thinking and opinions rather than 
having to accept a canon of 'accepted' or 'received' opinion. 

The A/S syllabus is divided into two major sections. Part I and Part II. Part II is sub-divided 
into two further sections: 

Parti 

Study of two texts and two films with a choice of one of the three following topics: Political 
and Social Issues, Detective Fiction, and Science Fiction (These three themes have been chosen for 
the initial curriculum. They will be changed regularly.) 
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Part II 
Section A 

A written diary covering a period of 4 nnonths recording personal reactions, feelings, 
aspirations in relation to the theme chosen and what the candidate has been reading or viewing or 
doing. 

Section B 

A choice of one or two pieces of work out of a choice of three topics, which is/are based on 
a subject or theme relevant to the chosen topic in Part I. 

What is novel for Hong Kong is that although Part I will be examined by written answers to 
written questions in the traditional manner, Part II consists of a portfolio of work divided into two 
sections: a diary and a project. This examination format means that there will have to be changes in 
teacher and student behaviour, changes which could well be beneficial to the tertiary institutions. 



Changes to Teacher and Student Behaviour 

This section of the paper outlines the changes to teacher and student behaviour whirh the 
new syllabus hopes to effect. Modern theories of composition described earlier will have their place 
in the techniques and strategies which will need to be developed by students and teachers as they 
deal with the changes inherent in the examination syllabus. 

It is worth noting that there is currently a reaction to a wholehearted ^process approach', one 
which states that product must not be neglected at the expense of process (Davies 1989). However, 
the reader will note that the writer has taken an unashamedly optimistic approach to the introduction 
of the new syllabus in the rest of the paper. No apologies are made for this approach. 

The section of the new syllabus which will have the greatest effect on teacher and student 
behaviour is Part II which encourages each student to respond in his/her own way to the theme that 
has been chosen. It also enables them to relate what they read, watch, encounter and perceive in 
the world around them to the theme that they are studying. 

The fact that each student portfolio, composed of diary and project, will be an individual 
product means that it must of necessity be different from that of their peers. This has far-reaching 
implications for the learning styles of students and for the classroom organization and 
teaching/learning strategies employed by teachers of this subject. 

In the first place students will NOT be able to receive a mass of notes which must be learned 
and regurgitated as classroom exercises and in the examination. Secondly, it will not be possible for 
students to borrow, plagiarise, or steal the work of students who have matriculated before them 
because the diary will be an individual record of reactions to contemporary events and how they 
relate to the theme being pursued. The record must be contemporary for the four months of 
record-keeping, although teachers will be encouraged to persuade students to keep a diary for a 
longer period of time than the four months stipulated in the regulations. (This, of course, has 
implications for the practice of writing.) In addition, students will be pursuing individual projects 
which will necessitate them working initially in groups and then alone under the SUPERVISION of 
their teacher, not as a whole group in one classroom. 
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For the teacher, a lock-step approach to the dissemination of 'knowledge' about the text will 
no longer suffice in Part II of the syllabus. Teachers will have to become counsellors, supervisors 
and facilitators who guide the students along their individual paths towards a greater understarxling 
of the theme and Its relation to the political/social worid around them. Their new role will include the 
creation of appropriate learning and discovery tasks for students, tasks which will help to gukle 
students to be able to think for themselves and to be analytical and critical. The new role will be 
perceived both as an exciting challenge and as a potential threat as teachers no longer become the 
repositories of an accepted canon of knowledge. They will have to become aware that students 
working on a project will eventually *know' a lot more about the project than they do. It will be an 
opportunity for teachers to behave like readers in the non-school worid - when they approach texts, 
when they (the teachers) are non-knowers. This Is quite different from their normal role where they 
often know what is right and what the students should be writing about even before the text is 
written. 

Guidelines for teachers, being developed at the moment by the Curriculum Development 
Council/Hong Kong Examinations Authority joint working party, make it clear that students in the 
sixth form will not be classroom bound all the time during the periods devoted to this subject. Time 
will be made for them to work in the library on individual projects. They will be expected to carry 
over from the classroom an awareness of how the theme relates to all that they see, read and do. In 
addition, time will have to be spent on individual planning, research, drafting and the revision of 
ongoing work on the project. Teachers will need to become aware of new methods of classroom 
management in order to implement the new curriculum. Whole group teaching for much of the time 
will disappear. 

These changes augur well for attitudes to learning and study when these students eventually 
arrive in their chosen institutions. 



Implications of These Changes for Tertiary Institutions in Hong Kong 

The implications of these changes to teaching and learning styles in the upper forms of 
those secondary schools in Hong Kong which decide to opt for the A/S level English Literature 
syllabus could be significant in terms of the quality of student thought and writing which occurs as a 
result of these developments. 

It is pc;.sible that students will learn a number of new study skills. They will have to learn 
how to use the library efficiently, how to use references, how to skim and scan texts in a real 
situation and not merely in the section of the Use of English textbook used in the language 
classroom, how to relate what they are doing in class to the worid around them, how to acquire new 
knowledge by fitting it into existing schema, and, most importantly of all: 



These changes, which could be forced upon students by the most powerful agent for 
change in Hong Kong - the examination syllabus - will be an excellent precursor for the work that 
students will encounter in the tertiary institutions. Indeed, it may well have spin-off for teaching and 
learning within the institutions themselves as the expectations of the students are raised. 



- how to work on their own on tasks set by and agreed with their 
teachers 



- how to develop their skills in thinking, planning, organising, 
selecting and learning. 
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At the level of writing sl<il!s development, the new syllabus has the potential for improving 
student writing in keeping with the beliefs of a process approach to writing. The sheer amount of 
writing that students will be encouraged to do could benefit the quality of Uie writing although 
Krashen (1984) has sounded a cautionary note that quantity may not guarantee quality. Students 
will be encouraged to go through the natural 'processes' of writing which were described above 
The advocates of a process approach would claim that with the opportunities for extensive planning 
and research, a macro-structure of the project will be created by the students under the supervision 
of the teacher. In addition, the potential for increased levels of coherence will be available as 
students become aware of how the macrostructure affects the 'texture' of the piece where coherence 
(overall unity of the text), cohesion (connectivity at the sentence and paragraph level), and 
thematisation (what follows on from the previous sentence) combine together to produce a 
well-developed, logical piece of writing, where there is every opportunity for thinking about the 
processes of revision and re-writing in order to improve earlier drafts. 

It should be noted, however, that some notes of caution have recently emerged with writers 
commenting on some of the drawbacks of a fully-fledged process approach (e q Zamel (1987) and 
Moore (1987) quoted in Kelly (1989)). \ i u 

As described above a major change will occur when the teachers read the diaries and 
project drafts of the students. These pieces of text will be real texts, not mere examination answers 
written for a faceless examiner. Teachers will have to act as genuine readers, learning about the 
Ideas that the young writers are creating, developing, organising, connecting and explicating This is 
an important point because the students will be in 'control' of their own diaries not writing to a 
teacher-conceived task. In addition the teacher will not know what to expect. The teacher will thus 
much more resemble a reader coming fresh to a text. The teacher will, in effect, be a 'non-knower'. 

Prescribed answers will no longer have a place in Part 11 of the new syllabus. This can be 
seen as a very welcome trend away from the 'model answers' which abound in Hong Kong's 
bookshops. The emphasis will be on the individual response of the students. The strategics 
described by researchers such as Elbow, Raimes, Faigley and Witte and all other adherents of 
process approaches will be evidenced as students plan, organize, develop ideas, draft and revise in 
Section B, and where they produce individual responses when they 'own' their own text in Section A 



A Final Cautionary Note 

If readers feel that all the above sounds too good to be true, they may well be correct in 
terms of the numbers of prospective candidates who prepare for the option - but not in the potential 
effect of the new curriculum on those students who do decide to opt for the English Literature A/S 
level. ' 

Those who choose the new syllabus are unlikely to be disappointed, are likely to get a great 
deal of personal insight out of choosing the option, are likely to be better writers at the end of the 
process than those who do not have the opportunity to be exposed to the learning that will 
accompany the option, and will undoubtedly be better prepared for tertiary level studies than manv 
of their peers. ' 
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Errata (Issue 13) 

P. 75, in Table 2: The correct value for sentence structure is "27", and not "1". 

P. 78: The penultimate sentence should refer to "cases such as V,9", and not to "cases such as V.10". 
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